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Executive Summary 


The Special Education Technology Integration Specialist (SETIS) program provides 
professional development for special education teachers to assist them in achieving profi- 
ciency with 21st Century Technology Tools. In 2011-2012, its 7th year, the program trained 
16 special educators as models, coaches, and mentors of technology integration at schools 
and within classrooms. This study examines SETIS program implementation, use, and im- 
pact across three key stakeholder groups: SETIS, teacher colleagues, and school administra- 
tors. 


Method of study 


SETIS candidates were surveyed once, using a retrospective pre-post survey adminis- 
tered at the conclusion of the school year. Teacher colleagues and school administrators, 
identified and invited by SETIS due to their close working relationships, participated in pre- 
post surveys administered at the beginning and ending of the school year. 


Findings 


The program is successfully equipping SETIS with the capacity needed to implement 
technology integration in schools and classrooms, as evidenced by significant differences in 
mean scores and large to very large effect sizes in the SETIS retrospective pre- and postpro- 
gram self-ratings. Teachers indicated SETIS activities led to increases in coteaching among 
teachers and SETIS, improved technology integration in classrooms, raised technology 
knowledge among teachers, and enhanced student experiences. School administrators re- 
ported greater student engagement as a result of integrating technology into their classwork. 
Teacher colleagues and school administrators reported leveraging SETISs’ skills and re- 
sources in the ways they anticipated. SETISs named administrative support as the most 
common factor in facilitating meaningful collaboration with teachers. Program barriers were 
perceived by SETISs and school administrators as moderate. SETISs reported a lack of time 
as their largest barrier; computer access for students, and internet speed were also primary 
concerns. Survey results revealed 25% of the participating administrators were not aware a 
SETIS would be present in their schools at the beginning of the school year. 


Limitations of study 


Relying upon self-reported information carries the risk of response bias. Among 
teachers and administrators small sample sizes and the inability to track response rates or 
match pre- and post-survey results were also limitations. 


Recommendations 


With the capacity to train 25 SETISs per year and increasing technological demands 
in classrooms, program staff are urged to recruit more SETIS candidates. Other recommen- 
dations include encouraging SETIS candidates to conduct more staff development at their 
schools; providing SETISs expanded opportunities to work together in face-to-face settings, 
to help them more effectively implement technology integration within the specialized con- 
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tent of special education; improving communication at all program levels to ensure greater 
awareness of SETISs’ presence in schools and the optimal use of their skills and resources; 
promote scheduling that allows teachers and SETISs time to cocreate technology-integrated 
lesson plans; and incorporate mechanisms in future evaluations that will allow for tracking 
and matching of teacher and administrator responses in pre- and postprogram surveys. 
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Introduction 


The Special Education Technology Integration Specialist (SETIS) program was creat- 
ed through an expansion of the Technology Integration Specialist (TIS) program. The origi- 
nal TIS program was designed to provide and coordinate professional development activities 
for all teachers and administrators on a school-level basis to assist them in becoming profi- 
cient in the use of 21st Century Technology Tools—a key component of Teach 21.! The TIS is 
an individual who models, coaches, and mentors teachers in using statewide technology re- 
sources to meet West Virginia’s content standards and objectives. The TIS also assists in the 
implementation of county and school technology plans, and in the use of other county and 
school software applications. 


The West Virginia Department of Education piloted the TIS program through Part D 
of Title II technology funds provided by the United States Department of Education. The 
program has now been expanded to include library media specialists, special educators, Title 
I teachers, career and technical education teachers, and county- and school-based TISs who 
are supported through local county funds. Special educators were brought into the TIS pro- 
gram in April 2006; they are referred to as special education technology integration special- 
ists, or SETISs. 


This study will examine implementation, use, and impact of the 2011-2012 SETIS 
program. To this end, findings from surveys of three critical stakeholder groups will be dis- 
cussed: (a) SETISs themselves, (b) teachers who either cotaught with or whom a SETIS had 
some degree of influence with, and (c) administrators at schools participating in the SETIS 
program. 


Currently in its 7th year, each round of the SETIS program runs from early summer 
of one year to late spring of the following year; participants have until late summer to com- 
plete program requirements. Thus, the program allows for training prior to the beginning of 
a school year and concludes after the school year has ended. At most, 25 participants are se- 
lected for each cohort. TIS program roles and responsibilities (also applicable to SETIS pro- 
gram participants), as stated in the TIS Assurance Statement and Agreement (Office of 
Special Programs, 2011), include the following: 


e participate in and successfully complete 40 days (320 hours) of required profes- 
sional development (including both online and face-to-face sessions); 


e use acquired technology integration skills to improve instruction for both stu- 
dents and educators; 

e use enhanced knowledge and skills to build effective consultative and coteaching 
relationships with all teachers; 


e use collaborative planning time to assist in integrating technology into the in- 
structional units being planned by teachers; 


1 The other two components of Teach 21 are 21st Century Learning Skills and 21st Century 
Content Standards. For more details about Teach 21, visit http://wvde.state.wv.us/teach21/. 
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e serve as a resource to educators on technology integration, as appropriate, in the 
implementation of information literacy, independent learning, and social respon- 
sibility; 

e communicate the importance of developing 21st century skills for all students, in- 
cluding those with disabilities, to improve academic achievement and postsec- 
ondary outcomes; and 


e continue to teach students and educators in West Virginia public schools as a TIS 
in consultative and coteaching relationships for 2 years after completion of the 40 
days of required professional development. 

Participants in the TIS program receive a laptop computer and camera for their 
school from the WVDE and additional technology resources including white boards, LCD 
projectors, scanners, and color printers from their county office. Additionally, participants 
receive the equivalent of 40 days (320 hours) of professional development and a $5,000 
grant to help defray the costs of the TIS professional development. Grant funds can be used 
to pay the costs for stipends, substitute teachers, travel expenses, conference registration 
fees, and other costs associated with teachers’ participation in the TIS program. Upon com- 
pletion of the program, participants may apply for a credential in instructional technology 
integration awarded by the WVDE Office of Professional Preparation. 


Rationale for Study 


In collaboration with the WVDE Office of Instructional Technology, Office of Special 
Programs, and Office of Career and Technical Instruction, the Office of Assessment, Ac- 
countability, and Research? (OAAR) developed an evaluation plan to assist the WVDE in de- 
termining the impact of the TIS program on selected TIS applicants, participating schools, 
teachers, and students (OAAR, 2010). The current study of the SETIS program covers sever- 
al components of that 2010 TIS program evaluation plan. Moreover, it expands upon the 
2010-2011 SETIS program teacher and administrator reports3, focusing on use, utilization, 
and impact of technology integration in 2011-2012 among three key stakeholder groups: 
SETIS, school administrators, and teachers/coteachers. While this is the second examina- 
tion of administrator and teacher perceptions about the SETIS program, it is the first time 
findings from all three stakeholder groups have been consolidated into one report. 


Evaluation Questions 


Drawn from the TIS evaluation plan, the SETIS program adopted five overarching 
evaluation questions: 


2 The Office of Assessment, Accountability and Research has since divided into two separate 
offices—the Office of Assessment and Accountability (OAA) and the Office of Research (OR). The Of- 
fice of Research was charged with implementing the evaluation plan described in this report. 


3 Access these reports at 
http://wvde.state.wv.us/research/reports2012/WVSpecialEducationTISProgramTeachersReport2012.pdf and 
http://wvde.state.wv.us/research/reports2012/WVSpecialEducationTISProgramAdministratorsReport2012.pdf. 
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Introduction 


To what extent does the TIS program build the capacity of participating TISs to 
plan and facilitate (a) teaching and learning, (b) information access and delivery, 
and (c) program administration? 


To what extent do TISs encounter barriers to successful program implementation 
(e.g., financial, temporal, relational, etc.)? 


To what extent is the level of technology integration in TIS schools positively im- 
pacted through participation in the program? 


In what ways have school administrators and teachers leveraged the TIS and the 
resources provided by the TIS? 


What impact has the TIS program had on students’ technology literacy in partici- 
pating schools? 


This study, analyzing survey results from SETISs, teachers, and administrators, ad- 
dresses each evaluation question. The SETIS Survey, a retrospective survey conducted at the 
end of the school year, concentrates on EQs 1 and 2, and assists with examining EQ4. The 
teacher surveys, administered at the beginning and end of the school year, focus on EQ3 but 
also touch upon EQs 4 and 5. The administrator surveys, also conducted at the beginning 
and end of the school year, focus primarily on EQ4 and collect additional evidence to ad- 
dress EQs 2 and 3. 
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Methodology 


Preintervention and postintervention surveys were administered to teachers and 
school administrators in the study at the beginning and ending of the school year. Special 
education technology integration specialists (SETISs) were surveyed once, using a retrospec- 
tive pre-post survey administered at the conclusion of the school year. All surveys were con- 
ducted using a secure on-line delivery and collection service. 


Participant Characteristics 


Three distinct groups were included in this study: SETISs, teacher colleagues, and 
school administrators. SETISs are special education teachers who were accepted into the 
SETIS program and received specialized training. The 2011-2012 SETIS cohort consisted of 
16 special education teachers, working in schools at all three programmatic levels (i.e., ele- 
mentary, middle, and high school), located in 11 school districts throughout the state. In this 
study, teacher colleagues are defined as teachers who cotaught with SETISs or with whom 
SETISs believed they had the greatest influence regarding technology use. School adminis- 
trators were defined as principals, assistant principals, or other teachers who serve in an 
administrative capacity in the schools where SETISs served. 


Sampling Procedures 


All 16 SETISs in the 2011-2012 cohort were invited to participate in this evaluation 
study. SETISs were responsible for selecting and recruiting teacher colleagues and school 
administrators to complete surveys at the beginning and end of the 2011-2012 school year 
(Pre and Post Surveys). This method of sample selection precludes our ability to report re- 
sponse rates, as SETISs were not required to report the number of administrators and teach- 
er colleagues they initially invited for each of the surveys or how many times they reminded 
invitees to participate. It should be noted that among the teacher colleague and administra- 
tor survey respondents, some counties were represented in one survey but not the other. Re- 
spondents from schools in these counties were removed from the sample before analysis to 
ensure that the Pre and Post Survey results were as comparable as possible. 


Measures and Covariates 


All evaluation questions in this study were addressed through the use of online sur- 
vey instruments designed to solicit perceptions of the three stakeholder groups mentioned 
above. The SETISs completed one survey, the Special Education Technology Integration 
Specialist (SETIS) Survey (see Appendix A, page 51). Teacher colleagues completed the 
SETIS Teacher Pre Survey and SETIS Teacher Post Survey see, page 63), while administra- 
tors completed the SETIS Administrator Pre Survey and SETIS Administrator Post Survey 
(see Appendix C, page 77). The WVDE Office of Research developed the instruments in col- 
laboration with WVDE Office of Special Programs staff. More information about each of 
these instruments follows. 


Methodology 


Special education technology integration specialist survey 


The SETIS Survey, designed to capture the SETIS experience, is a questionnaire that 
included a variety of questions, including retrospective pre/post items that asked respond- 
ents to reflect on conditions before they participated in the program and then reflect again 
on the those conditions at the end of the program. The survey was administered at the end of 
the 2011-2012 school year. 


Using multiple choice and open-ended questions, the instrument collected demo- 
graphic information about respondents and examined four types of capacity building, using 
four distinct indices: human capacity, organizational capacity, structural capacity, and 
material capacity. The capacity indices were based on definitions found in Determining Ca- 
pacity within Systemic Educational Reform (Century, 1999). The human capacity index 
measures intellectual proficiency (knowledge, expertise, and understanding) and will (inter- 
est, patience, and persistence). Items in this index were used to determine what, if any, 
change SETIS’s perceived in their own capacity over the course of the program. The organi- 
zational capacity index focuses on the interaction, collaboration, and communication among 
individuals within a system; in this case the system is the SETIS program and the individuals 
are the SETISs and the teachers and school administrators they work with. The structural 
capacity index includes elements such as policies, procedures and formalized practices; ele- 
ments that are part of the system (i.e. the SETIS program), but also independent and not 
under the direct control/influence of those using the elements. The material capacity index 
is concerned with fiscal and/or other material supports available to people (e.g. SETIS can- 
didates) within a system (e.g. the SETIS program). The open-ended questions provided an 
opportunity for respondents to expand upon the capacity indices as well as describe attrib- 
utes they found most valuable, what they would change, and barriers encountered in the TIS 
program. 


Employing a 5-point Likert-type scale of agreement—including 1 (strongly disagree), 
2 (disagree), 3 (neutral), 4 (agree), and 5 (strongly agree)—each of the four capacity sec- 
tions prompted SETISs to consider their level of agreement with a statement before and af- 
ter participating in the TIS program. For example: 


Before participating in the TIS program. . . ... [model(ed) lessons that integrate tech- 
After participating in the TIS program... nology for other teachers in the school 


The human capacity index asked SETISs to consider their knowledge, skills and effi- 
cacy with technology integration. The organizational capacity index included survey items 
about the extent of interaction SETISs had with others in their school/county to effectively 
integrate technology, and how their practices have changed due to participation in the TIS 
program. The structural capacity index consisted of items measuring the extent policies, 
procedures, and practices within the SETIS’s school/county have changed as a result of hav- 
ing a technology integration specialist. The final index, material capacity, included survey 
items concerning access to resources necessary to accomplish technology integration in their 
school. 
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Demographic information collected included the county where the SETIS is located, 
their role within their school, the programmatic level of their school (i.e., pre-k, elementary, 
middle, high), and if their school currently employed any other person who had completed 
the TIS program. 


Teacher Pre Survey and Post Survey 


The surveys included multiple-choice and open-ended questions. Demographic data 
collected from the teacher colleagues included school district, school name, primary role 
(i.e., teacher, aide, etc.), years of teaching experience, and years of coteaching with a SETIS. 
Respondents were also asked about the education environment for the majority of their stu- 
dents. Described in detail below, multiple-choice response items were used to assess teacher 
colleagues’ technology use and utilization. Open-ended questions allowed respondents to 
provide a descriptive account of the roles of SETISs, what the respondent hoped to accom- 
plish (Teacher Pre Survey), or has accomplished (Teacher Post Survey) by working with a 
SETIS, and any additional comments they chose to provide. 


The technology use items in the surveys were based on a 4-point scale to indicate 
how often respondents performed certain technological practices, such as “I use information 
from digital sources to promote learning and engage students in classroom activities,” with 1 
(not usually), 2 (sometimes), 3 (often), or 4 (almost always) as response options. Survey 
items were grouped into five indices based on the TIS Professional Development Model4 
(NETS-T, 2008): (a) designing and developing digital-age learning experiences and assess- 
ments, (b) modeling digital-age work and learning, (c) engaging in professional growth and 
leadership, (d) promoting and modeling digital citizenship and responsibility, and (e) facili- 
tating and inspiring student learning and creativity. For the purposes of this report, the indi- 
ces are labeled as Design, Model, Engage, Promote, and Facilitate. 


Administrator Pre and Post Surveys 


In the Administrator Pre Survey, administrators were asked to respond to six items 
prospectively, concerning how they anticipated using the SETIS’s services; in the Adminis- 
trator Post Survey, they were asked to provide an assessment of how they actually utilized 
the SETIS services. The Post Survey also included one question asking school administrators 
if the presence of a SETIS in their school could be linked to increased student engagement in 
curricular activities. Both surveys employed a 5-point, Likert-type scale; the Pre Survey scale 
was based on likelihood with 1 indicating unlikely and 5 indicating likely whereas the Post 
Survey used a scale of agreement where 1 denoted untrue and 5 signified true. Due to the 
nature of the surveys, the Pre Survey asking administrators to measure intended implemen- 
tation and the Post Survey asking about actual leverage, it was necessary for the wording of 
the items to differ slightly. For example, a Pre Survey item would read, “I plan to ask my 
SETIS to share what he/she has learned by leading standards-based professional develop- 
ment for the other teachers in my school,” and its Post Survey counterpart would be worded, 
“My SETIS shared what he/she has learned by leading standards-based professional devel- 
opment for the other teachers in my school.” 


4 Based upon the National Educational Technology Standards for Teachers (NETS-T) 
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Administrators were also asked to provide descriptive accounts, via open-ended sur- 
vey items, about their expectations concerning the role of a SETIS, and what they expect to 
realize (Pre), and actually realized (Post) from having a SETIS in their schools. Demographic 
data, such as the role of the respondent in their school and length of time in that role, were 
also collected. 


Research Design 


The SETISs completed their retrospective pre/post survey at a face-to-face meeting 
held in May 2012. In attendance were 14 SETISs in this year’s cohort, and a coordinator 
from the West Virginia Department of Education. 


The teacher colleague and school administrator surveys were each administered 
twice to assess the extent to which teachers and schools expected and actually utilized the 
services of the SETIS. Among the teacher colleague cohort, Pre Survey data collection took 
place from August through November, 2011. The data collection period for the school ad- 
ministrator Pre Survey occurred from August through October, 2011. Both Post Surveys were 
open for data collection from May through June, 2012. 


Data Analysis 


Both quantitative and qualitative data analysis techniques were used to address the 
data collected for this study. To analyze quantitative survey items (i.e., multiple choice ques- 
tions), we used SAS 9.2 to produce descriptive statistics including frequencies (i.e., percent- 
ages), and to calculate and interpret measures of central tendency and dispersion (i.e., 
means and standard deviations). Statistical significance was established by conducting t 
tests; a t test determines if the difference in mean scores between two groups (such as pre- 
test and posttest) represents a pattern and is not simply the result of chance. Two types of t 
tests were applied to the data in this study, paired-samples and independent-samples t¢ tests. 
It was possible to use paired-samples t tests for the SETIS Survey results. Paired-samples t 
tests consist of matched pairs; since the SETIS Survey is a retrospective survey we know that 
the before-and-after results from each participant are a matched pair. However, it was im- 
possible to link pre survey results to post survey results with the Teacher and Administrator 
surveys. Therefore, these surveys required the use of independent-samples t tests. Pre and 
Post Survey means were considered significantly different when t tests yielded p values of 
0.05 or less. A p value of 0.05 or less indicates a 95% probability that the survey results were 
not observed due to chance. Additionally, effect sizes using Cohen’s d were calculated. Akin 
to their name, effect sizes measure the magnitude of difference in the mean scores between 
two measurements, in this study the Pre and Post Surveys. Further, as a measure of strength 
between the differences of two mean scores, effect sizes are not affected by sample size. Co- 
hen (1988) interpreted effect sizes lower than 0.15 to be negligible, between 0.15 and 0.40 to 
be small, between 0.40 and 0.75 to be medium, between 0.75 and 1.10 to be large, and above 
1.10 to be very large. Effect sizes were included among statistically significant survey results 
only; if a result is not statistically significant, then further analyses are not warranted. 


Statistical analysis was also conducted to test internal consistency among survey 
items intended to measure the same concept. For example, the surveys used to inform this 
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study use multiple survey items to construct the indices employed in the SETIS and Teacher 
Surveys. Internal consistency is most often measured with Cronbach’s alpha (aq), a reliability 
coefficient that measures the correlations between different items on the same index. An al- 
pha of 0.70 or higher is considered reliable; Alphas above 0.60 may be reliable, but ques- 
tionable; and generally, Alpha scores below 0.59 are not considered reliable. 


Qualitative responses (i.e., responses to open-ended questions) were read, reread, 
organized, and coded according to broad themes, taking into consideration that respondents 
sometimes mentioned more than one theme when responding to a question. The Results 
section includes tables comparing Pre and Post Survey results and narration describing the 
themes. The qualitative data included in each of the three surveys (SETIS, Teacher, and Ad- 
ministrator) in this study were designed to enrich our data by providing respondents the op- 
portunity to answer questions using their own words. 


The West Virginia Special Education Technology Integration Specialist (SETIS) Program | 9 


10 


Results 


Special Education Technology Integration Specialists Survey 
Demographics 


Of the 16-member 2011-2012 cohort, 14 special education technology integration 
specialists (SETISs) responded to the SETIS Survey—an 87.5% response rate. Survey re- 
spondents were from 11 counties; two counties had multiple SETISs participate in the sur- 
vey. 


SETIS were asked about their role in their school (Table 1). Of the 14 respondents 
71.4% (10 of 14) answered that they were classroom teachers. The remaining four respond- 
ents (28.6%) chose the other category; when prompted to specify, three stated that they were 
special education teachers, and one responded he or she was a gifted teacher. 


Respondents were also asked to indicate the programmatic level of their school. The 
responses show that the SETISs participating in the survey represented all school program- 
matic levels. Middle schools, with over 40% (six of 14 respondents), had the highest level of 
representation, while elementary schools came in second with just over 21% (three of 14). 
Two (14.3%) SETISs chose both elementary and middle school as their schools’ program- 
matic level. The following three programmatic levels were chosen once (each representing 
7.1% of the responses): high school, pre-k and elementary, and pre-k through high school 
(Table 1). 


The SETISs were also asked if, to their Table1. | SETIS Respondent Demographics 


knowledge, their school currently employed Number 

any other individual who had completed or is responses Percent 
currently completing the Technology Integra- Characteristic (n=14) _ response 
tion Specialist (TIS) program. To this, almost — Roleinschool 
65% (nine of 14) responded yes, and 35% (five Total 14 100 
of 14) responded no. It is important to note !@8sroom teacher 10 iio 
that these TIS program participants may have @™inistrator e oo 
been either a TIS or a SETIS. Sunperistat r of 


Other 4 28.6 


Near the end of the survey, SETISs Programmatic level ¢ 00 
were asked if they had conducted any formal Total 14 100 
staff development related to technology inte- Elementary 3 
gration at their school within the past year. Of Middle 6 
the 14 SETISs participating in the survey, 12 High 1 7.1 
responded to this question. Only two, (16.7%) Pre-K and elementary 1 71 
reported that they had conducted formal staff lementary and middle 2 14.3 
development. When asked to describe the _Pre-K-High 1 ee 
staff development, the two respondents mentioned online enrichment resource training for 
parents, and instruction for special education teachers on the use of iPads during testing. 
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Building capacity indices 


To address EQi—that is, To what extent does the TIS program build the capacity of 
participating TISs to plan and facilitate: (a) teaching and learning, (b) information access 
and delivery, and (c) program administration?—the quantitative sections of the SETIS Sur- 
vey were aggregated to create four capacity indices: human, organizational, structural, and 
material. Each capacity index was composed of multiple survey items. As described earlier, 
each of the four capacity sections prompted SETISs to consider their level of agreement with 
a statement before (pre) and after (post) participating in the TIS program. Figures 1—4 illus- 
trate the pre- and post-SETIS program means by individual survey item in each index. Fig- 
ure 5 presents pre and post standardized capacity indices where responses to individual 
survey items were used to create mean scores for each index. It is important to note that the 
items and indices are based on a 5-point Likert-type scale, with the possible responses rang- 
ing from a low of 1 (strongly disagree) to a high of 5 (strongly agree). 


Human capacity, the first index in the survey, consists of 17 items measuring the in- 
tellectual proficiency and will of the SETISs’ themselves. The items are: 
1. I have a strong understanding of how to use/apply the resources from Thinkfinity. 


2. I have a strong understanding of how to use/apply the resources from Curriculum 
Pathways (SAS). 


3. I have a strong understanding of how to use/apply the resources from West Virginia 
Writes. 


4. I use digital resources and tools to make assignments for students that are based upon 
their individual interests, abilities, and learning needs. 


5. Ihave a strong understanding of how to use/apply the resources from TechSteps. 
6. I am able to assess the quality and legitimacy of web resources. 


7. I understand the most important issues surrounding legal use/copyright regulations 
and how they relate to integrating web resources and technology into lesson plans and 
instruction. 


8. I am able to identify the components of a URL and to ensure it is legitimate (e.g., pro- 
tocol, host, domain, directory, port address, etc.). 


g. I have a strong understanding of the core National Educational Technology Standards 
and Performance Indicators for Teachers (NETS-T). 


10. I have a strong understanding of the core National Educational Technology Standards 
and Performance Indicators for Students (NETS-S). 


11. I understand how to integrate Web 2.0 tools into instruction (e.g., podcasting, wikis 
and blogs, social networking, etc.). 


12. I integrate digital resources/tools into my work with teachers, students, and adminis- 
trators. 


13. I understand how to effectively integrate technology into instruction to improve the 
quality of students' educational experiences. 


14. I have a strong understanding of the county/school acceptable use policy. 
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15. I have a strong understanding of 21st century assessment. 


16. I have a strong understanding of how to design and implement project-based learning 
(PBL) in the classroom. 


17. I actively use action research to assess the impact of my teaching on student learning. 


To see what, if any, change SETIS’s perceived in their own human capacity over the course of 
the program, a pre- and post-SETIS program mean was calculated for each item (Figure 1). 
The items, consisting primarily of statements about understanding, use, and integration, 
each revealed statistically significant increases in mean scores from pre- to post-SETIS pro- 
gram. In fact, many items that had an average score of around 2 in the pre-SETIS program 
doubled their scores to 4 or higher in the post-SETIS program averages. The item that exhib- 
ited the largest change, from a pre average of 1.86 to a post average of 4.43, was Item 11, “I 
understand how to integrate Web 2.0 tools into instruction (e.g., podcasting, wikis and 
blogs, social networking, etc.).” Overall, the results indicate SETISs perceived significant 
gains in their intellectual proficiency and will. 


Figure 1. SETISs’ Retrospective Pre-Post Assessment of Growth in Human Capacity 
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N = 13 or 14, depending on the item 


Organizational capacity, the second index, consists of five survey items centered on 
the interaction, collaboration, and communication among individuals within the SETIS sys- 
tem (such as other teachers and school administrators). The organizational capacity items 
are: 


18. I model lessons that integrate technology for other teachers in the school. 


19. I collaborate with others within the school to effectively integrate technology into in- 
struction. 


20. I serve as a resource to other teachers regarding the effective use of technology in the 
school. 


21. I can effectively work with others to assess their learning and information needs (e.g., 
other teachers, students, administrators, etc.). 
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22. I have an ongoing dialogue with other staff members at the school about technology 
issues and how they can be addressed. 


Among the pre-SETIS program items, the ; ; 

i Figure 2. SETISs’ Retrospective Pre-Post 
means ranged from 2.29 to 2.86 (Figure Assessment of Growth in Organizational 
2). The averages increased significantly Capacity 
among all the post-SETIS program items, 
ranging from 4.07 to 4.54. The item that 
showed the largest increase was Item 19, “I 
collaborate with others within the school 
to effectively integrate technology into in- 
struction.” This item’s pre-SETIS program 
average was 2.29; the post-SETIS program 
average rose to 4.36. Overall, the compari- 
son of pre and post averages for each item 
reveals significant increases in organiza- 
tional capacity among SETISs in this pro- sein 
gram. N = 13 or 14, depending on the item 
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The next index, structural capacity, is composed of the following 17 survey items: 


23. The school has set a time and place where staff meet in professional communities of 
practice to discuss how to effectively integrate technology into instruction. 


24. Teachers at the school have time to co-plan and/or co-teach lessons that integrate 
technology into instruction. 


25. Staff at the school understand the acceptable use policy. 
26. Staff at the school understand key concepts and best practices regarding web literacy. 


27. Staff at the school understand the most important issues surrounding legal 
use/copyright regulations and how they relate to integrating web resources and tech- 
nology into lesson plans and instruction. 


28. Staff at the school are familiar with the technology components of the school's strate- 
gic plan. 


29. The school has policies and procedures in place that support the use of Project Based 
Learning (PBL). 


30. The school has policies and procedures in place that support the use of technology re- 
sources (e.g., Thinkfinity, Acuity, West Virginia Writes, etc.). 


31. Staff at the school regularly use Thinkfinity and or Curriculum Pathways (SAS) as a 
resource in the classroom. 


32. Staff at the school regularly use West Virginia Writes or another online writing pro- 
gram as a formative assessment of student writing. 


33. The school has a plan in place to support the implementation of TechSteps. 
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34. Staff at the school regularly use TechSteps as part of their core content instruction. 


35. Staff at the school understand the core National Education Technology Standards and 
Performance Indicators for Teachers (NETS-T). 


36. Staff at the school understand the National Education Technology Standards and Per- 
formance Indicators for Students (NETS-S). 


37. Staff at the school understand West Virginia's 21st Century Skills and Tools. 
38. Staff at the school use Web 2.0 tools for collaboration and instruction. 
39. Staff at the school frequently integrate digital resources/tools in their teaching. 


The structural capacity index includes elements such as policies, procedures, and formalized 
practices—that is, elements that are part of the SETIS program, but have been set in place 
independently of the program. Among some of the items in this index, increases in pre to 
post averages appear to be moderate, yet, all items demonstrated statistically significant in- 
creases (Figure 3). The item exhibiting the largest change, “Staff at the school frequently in- 
tegrate digital resources/tools in their teaching” (Item 39), rose from a pre average of 2.5 to 
a post average of 3.79. While all items rose significantly, this index revealed more conserva- 
tive increases among pre and post item averages than the other indices. 


Figure 3. | SETISs’ Retrospective Pre-Post Assessment of Growth in Structural Capacity 
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The final index, material capacity, is concerned with fiscal and/or other material 
supports available to people (e.g. SETIS candidates) within a system (e.g. the SETIS pro- 
gram). In this study, the material capacity index comprises the following five items focused 
upon technology resources: 


40. I have access to a variety of high quality technology resources at my school (e.g., com- 
puter, digital camera, whiteboard, etc.). 


41. I participated in a variety of professional development about integrating technology 
resources into instruction. 
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42. I feel I am able to direct staff in my school toward high quality technology resources 
that are relevant to their information needs. 


43. I have the ability to develop useful technology resources for my school that address 
our information needs. 


Figure 4. SETISs’ Retrospective Pre-Post Assess- 
ment of Growth in Material Capacity 
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All items revealed significant increases from 9.0 5 
the pre-SETIS program average to the post- 
SETIS program average (Figure 4). The item 
with the largest change was Item 44, “I 
know where to find useful and high quality 
technology resources that can be integrated 
into instruction.” This item’s pre-SETIS 
program average was 2.23, while the post- 
SETIS program average rose to 4.54. Over- 
all, the results indicate SETISs perceived 
considerable increases in material capacity 
throughout the SETIS program. 
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To measure perceived overall growth in capacity resulting from participation in the 
SETIS program, the items in each of the indices were averaged together to create standard- 
ized capacity indices. Illustrated in Figure 5, significant increases were observed in all of the 
indices from pre- to post-SETIS program involvement. The human capacity index showed 
the most dramatic increase, rising from a preprogram mean score of 2.32 to a postprogram 
mean of 4.23. Conversely, 
while still statistically signif- 
icant, the structural capacity 
™@ Pre SE TIS Program @ Post SE TIS Program index exhibited the smallest 
increase. 


Paired-samples ft tests 
were performed on each 
item as well as the standard- 
ized capacity indices; all dif- 

; oo - ferences were statistically 


significant (Table 2). There 
is a 95% certainty the differ- 
ences observed from the 
N = 13 or 14, depending on the item pre-SETIS program item 
responses to the post-SETIS 
program item responses are not observed due to chance. Additionally, Cronbach’s alpha (a), 
a reliability coefficient that measures internal consistency between different items on the 
same index (as with our capacity indices) was calculated for each index based on the prepro- 
gram survey items and postprogram survey items. An alpha of 0.70 is considered reliable; as 


Figure 5. SETISs’ Retrospective Pre-Post Assessment of Growth 
by Type of Capacity 
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seen in Table 3 all alpha’s from the preprogram and postprogram survey items are 0.713 and 
higher. 


Table 2. Standardized Capacity Indices: Pre and Post Means, T Test and Effect Size 


Pre Post Significance of difference = Cohen’s 
Index Mean SD Mean SD t df p d_ Effect size 
Human capacity 2.32 1.13 4.23 0.76 27.01 234 <.0001 2.07 very large 
Organizational capacity 2.54 1.10 4.29 0.60 13.13 67 <.0001 2.04 _—-very large 
Structural capacity 2.67 1.07 3.46 1.02 13.24 236 <.0001 0.79 large 
Material capacity 2.57 1.10 4.34 0.64 12.96 64 <.0001 2.04 _—-very large 


Table 3. Standardized Capacity Indices: Reliability Coefficients 


Index Itemsincluded PreSurveya Post Survey a 
Human capacity 1-17 0.925 0.915 
Organizational capacity 18-22 0.915 0.792 
Structural capacity 23-39 0.891 0.850 
Material capacity 40-44 0.713 0.841 


Effect sizes 


Effect sizes were computed to measure the magnitude of change from pre-SETIS 
program survey results to post-SETIS program survey results. Figure 6 illustrates the effect 
sizes observed among the capacity indices (see individual items results in Appendix D, page 
85). As mentioned in the Data Analysis section, effect sizes between 0.75 and 1.10 are inter- 
preted to be large, and those above 1.10 are considered very large. The results show very 
large effect sizes for the human, organizational, and material indices. The structural index 
exhibited the smallest effect size, however, at 0.79 it is still considered to be large. 


Deconstructing the capacity 
indices by examining the effect size of | Figure 6. SETISs’ Retrospective Pre-Post Assessment of 
each survey item permits us to see Growth: Effect Size by Capacity Index 
their individual strength and impact. 
As anticipated based on the effect siz- 
es observed in the capacity indices, 
very large effects were seen in all of 
the human capacity items (1-17), or- 
ganizational capacity items (18-22), 
and material capacity items (40-44). 
Effect sizes among the structural ca- m Effect size 
pacity items (23-39) ranged from 
0.46 (medium effect) to 1.46 (very large effect). 
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Overwhelmingly, individual item effect sizes within the indices were large to very 
large, with the exception of structural capacity, which contained eight items exhibiting me- 
dium effects (Appendix D, Table 24, page 88). Human capacity exhibited the largest effect 
size by index as well as by item. In fact, the three largest effect sizes by individual item were 
found in the human capacity index. Survey Item 10 (“I have a strong understanding of the 
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core National Educational Technology Standards and Performance Indicators for Students 
[NETS-S]”) yielded the largest effect size of all at 3.64. The second largest effect size, 3.6, 
was for Item 9 (“I have a strong understanding of the core National Educational Technology 
Standards and Performance Indicators for Teachers (NETS-T)”. With an effect size of 3.5, 
Item 11 (“I understand how to integrate Web 2.0 tools into instruction [e.g., podcasting, 
wikis and blogs, social networking, etc.”]) demonstrated the third largest effect size. The 
fourth largest effect size was found in the last item, 44 (“I know where to find useful and 
high quality technology resources that can be integrated into instruction”); part of the mate- 
rial capacity index, this item yielded an effect size of 3.46. 


Open-ended items 


There were five essay-style, open-ended items on the SETIS Survey. These questions 
were meant to capture qualitative information about the SETIS program and enrich the 
quantitative data discussed above. While a few SETISs did not respond to the open-ended 
items, the overall response rate was relatively high, ranging from approximately 70% to 
80%, depending on the item. 


The first question, “What would you consider to be the most valuable aspects of par- 
ticipating in the TIS program?,” received 11 responses. When analyzed, three dominant 
themes emerged among the answers: (a) learning about technology resources/tools, (b) 
practice using technology, and (c) technology integration. The most common theme, learn- 
ing about technology resources/tools, was cited nine times. Practice using technology was 
mentioned four times and technology integration twice (Table 4, page 20). Note that some 
responses may contain one or more comment as well as one or more theme, therefore, the 
frequency of themes often exceeds the number of respondents. The comments listed below 
are samples of the predominant theme, learning about technology resources/tools: 


The most valuable aspect of participating in the TIS program is learning about the 
abundance of technology-related resources, strategies, and tools available to teachers. 
It makes the teaching process much easier for us and much more enjoyable to our 
students. 


The wealth of knowledge that you gain about technology. The ability to learn from 
others in the TIS program as well as the leaders. The introduction to various aspects 
of technology...webinars, Twitter and others that normally I would probably not have 
participated in. 


The wealth of technology information. I have always strived to use technology and am 
wanting to find more innovative ways to use it in my classroom. This past year I have 
been able to get a printer that prints in color, a mimio, and have had access to iPads, 
and have been able to incorporate them all into many of my lessons. Students have 
absolutely loved the changes and the environment of the classroom has changed also. 
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Figure 7. SETISs’ Retrospective Pre-Post Assessment of Growth in Four Types of Capacity: Effect Size 
by Item Number 


WAN DU RWN PR 


Human capacity 


PRP RP RPP PRB 
NouwnPwnrrFr Oo 


a 
oO w 


Organizational 
capacity 


NN 
Re oO 


N 
NR 


Item numbers 


Structural 
capacity 


SS EES Ee 
a ee 
es ee ee 
re ee 
a ee ee 
a ee 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 


Material 
capacity 


1.5 2 


Effect size, Cohen’s d 


The West Virginia Special Education Technology Integration Specialist (SETIS) Program | 19 


Results 


Table 4. Valuable Aspects of TIS Program Participation 


Frequency of comments* 


Theme SETIS Survey (n = 11) 
Learning about technology resources/tools 9 
Practice using technology 4 
Technology integration 2 


*Some respondents provided more than one comment; therefore, the frequency 
of comments may exceed the number of respondents (n). 


The second question, “What would you change about the TIS program if you could?,” 
garnered nine responses. As seen in Table 5, five themes were identified. The most promi- 
nent theme, with four references, was program/content changes. This theme included sug- 
gestions about accountability, guidance, and course work such as the “blended learning 
activity.” Specific examples of this theme include: 


The only thing that I would possibly suggest is a reconstruction to the blended learn- 
ing activity. I know that I struggled with this part of the program, and found it very 
difficult to conduct it and incorporate it as thoroughly in the classroom as I would like 
to. For special education classes, even inclusion one, it’s difficult to do this, at least 
for me and at my school, due to the inability of many students comfort levels with the 
computer and the collaboration and support of the inclusion teacher is critical. 


I feel there needs to be another way of making people accountable rather than post- 
ing, reading, and replying on the discussion board. 


Maybe just a little more guidance along the way. That would help prod some of us to 

work! 

Two other themes highlighted logistic concerns; time changes were mentioned twice 
and suggestions about meeting changes were mentioned three times. Of the time sugges- 
tions, one stated that too much time was given to complete assignments, while the other said 
there needs to be more time. The meeting change theme included requests for an increase in 
face-to-face meetings and more SETIS-only meetings (i.e., fewer meetings with the entire 
TIS cohort). The theme no change was cited twice. One SETIS wrote, “Although it is a time- 
consuming and challenging endeavor, I think everything we did this year has value and 
should remain in place.” The last theme, N/A, was mentioned once. 


The third open-ended question Table5. Suggested Changes to the TIS Program 
asked the SETIS participants, “What struc- Frequency of comments* 
tures are in place at your school that allow theme SETIS Survey (n = 9) 
teachers to work meaningfully with the Program/content change 4 
TIS?” From the 11 responses, four core Time change 
themes were identified: (a) administrative Meeting change 
support, (b) common planning time, (c) No change 
TIS support, and (d) other (Table 6). Over N/A 
60% (7/11) of the SETISs referred to admin- “Some respondents provided more than one comment, 
istrati “4 he leadi t t l therefore, the frequency of comments may exceed the 
istrative support as the leading structural 5 mber of respondents (n). 
support. Common planning time, with four 
mentions, was the second leading response. The other themes, TIS support and other were 
each mentioned once. Below are examples of the dominant theme, administrative support: 


PNM WN 
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For the most part, there is a great deal of 
support from the administration. 


There is definite support from the admin- 
istrators and teachers are interested. 
There is time during in-service days to 
work together. 


I have a lot of support from my admin- 
istration, but their hands are often tied by 
scheduling and “higher ups” I know that 
next year, both will be working to give me 
a better schedule to work with teachers 


Results 


Table 6. School Structures Supporting TIS and 
Teacher Collaboration 


Frequency of comments* 


Theme SETIS Survey (n = 11) 
Administrative support 7 
Common planning time 4 
TIS support 1 
Other 1 


*Some respondents provided more than one comment, 
therefore, the frequency of comments may exceed the 


and students. 


number of respondents (n). 


The fourth open-ended question asked SETISs to describe ways they worked with the 
administrators at their school to address technology related issues or concerns. Eleven 
SETISs responded to the question and five overarching themes were discovered in the re- 


sponses (Table 7). Over 35% (4/11) 
of the SETISs stated that they dis- 
cussed technology resources and 
needs with school administrators. 
Three other themes, (a) provide 
training, (b) collaborate with 
teachers, and (c) provide specific 
assistance were each represented 
twice. One SETIS cited none, as they 
did not have the time to work with 
school administration. The following 


Table 7. Addressing Technology Issues/Concerns: SETIS 
Support to School Administration 


Frequency of comments* 
Theme SETIS Survey (n = 11) 


Provide training 2 
Collaborate with teachers 
Discuss resource/technology needs 
Provide specific assistance 


PN BN 


None 


*Some respondents provided more than one comment, therefore, the 
frequency of comments may exceed the number of respondents (n). 


are key examples from the identified themes. 


Discuss resource/technology needs: 


In every way. We have consulted on scheduling, student needs, staff needs, software 
upgrades, AUP, TechSteps, online testing. 


Spoke with administrator to discuss technology needs and access. 


Collaborate with teachers: 


I was able to get my schedule changed slightly so that I could do more collaboration 
with teachers and work directly with students. During teacher meetings, I was given 
the opportunity to share with my staff resources and ideas about how to integrate 


technology in instruction. 


Provide training: 


I set up a plan for next year in providing technology training on specific programs I 
know and have used in the classroom successfully. 


The final open-ended question asked respondents, “What barriers, if any, did you en- 
counter in implementing what you learned as part of the TIS program?” Eleven SETIS par- 
ticipants provided answers and five themes became evident during data analysis (Table 8). 
The lack of adequate time appears to be the most frequent barrier with four of the 11 SETISs 
identifying this theme. One example is found in this statement: “There is not enough time to 
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implement everything we learn, especially with the demands of fulfilling our requirements 
for Acuity, West Virginia Writes, and TechSteps.” 


The second most common fables. Barriers Encountered by SETISs 


theme, access to/quality of tech- Frequency of camments 
nology, was mentioned three times. theme SETIS Survey (n = 11) 


SETISs voiced concern about access Time 4 
to computers and computer labs, as Common planning time 

well as bandwidth issues. The lack Access to and quality of technology 
of common planning time among Collaboration 

SETIS and their co-teachers was N/A 


PF PWhN 


referred to twice. The lack of col-  *Some respondents provided more than one comment, therefore, the 
laboration between SETIS and frequency of comments may exceed the number of respondents (n). 


general education teachers was mentioned once. Finally, one SETIS responded N/A. 


Teacher Pre and Post Surveys 


Teacher colleague demographics 


In all, there were 48 respondents to the Special Education Technology Integration 
Specialist (SETIS) Teacher Pre Survey (hereafter, Teacher Pre Survey) and 28 to the Special 
Education Technology Integration Specialist (SETIS) Teacher Post Survey (hereafter, Teach- 
er Post Survey). However, when the data were 


Table 9. County Location of Teacher 
cleaned, there were 40 Teacher Pre Survey responses 


Respondents’ School 


and 27 Post Survey responses used in the analysis. 


. Pre Post 
Among the eight Pre Survey responses excluded from 5.040, (n = 48) (n = 28) 
analysis, one was blank, one was completed by a Total 18 28 


school administrator, and six were from counties that Barbour 3 
had zero representation in the Post Survey. Only one Boone 


1 

Post Survey was excluded due to not having county Greenbrier 5 1 
representation in the Pre Survey. It is also important Hampshire 17 14 
to note that SETISs may have invited one teacher col- Harrison 3 3 
league, or several teacher colleagues to complete the 4ackson” 1 0 
survey. Currently, there is no way to track how many Marion” Q : 

bes uc . Mineral 4 1 
teacher colleagues were invited to participate in the eae > 4 
survey. Therefore, it is not possible to calculate a re- Raleigh i : 
sponse rate. Starting with the 2012-2013 SETIS co- Randolph 6 3 
hort, matching Teacher Pre and Post Survey  Tycker* 1 0 
respondents will be possible. While the new approach — Upshur 1 1 


will not solve the issue of determining response rate, *Counties excluded in data analysis due to lack 

it will allow us to know if the same teachers who re-__ °F Fepresentation in either the Pre Survey or the 
‘é Post Survey. 

sponded to the Pre Survey are also responding to the 

Post Survey. 


Teacher colleagues were asked to indicate the county location of their school. In the 
Teacher Pre Survey they cited 12 different counties; only nine counties were selected in the 
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Teacher Post Survey. As seen in Table 9, the counties with zero surveys in either the Pre or 
Post Survey were excluded from further data analysis due to a lack of equal geographic rep- 
resentation. 


In addition to the location of their school, teacher colleagues were also asked a series 
of questions about the length of their teaching and SETIS coteaching experience, their role 
within their school, and the educational environment of the majority of their students. For 
analysis, only teacher surveys from counties that had representation in both the Teacher Pre 
and Post Surveys were included; this criterion resulted in analysis of 40 Pre Survey respons- 
es and 27 Post Survey responses. 


While there were more respondents to the Teacher Pre Survey than the Teacher Post 
Survey, demographic analysis suggests both groups had like characteristics, see Table 10. 
Comparing Pre and Post Survey re- 
spondent demographics, the teacher col- 
leagues reported similar roles, student 
general/special education populations 


Table 10. Teacher Colleague Survey Respondent 
Demographics 


Pre Post 
percent percent 


taught, and years of teaching experience. Characteristic (n = 40) (n =27) 
Nearly all respondents to both surveys [TT | GilNNSeINSSeIE l/l 
were teachers, who were about evenly Teacher 975 926 
divided between special and general edu- Aide 0.0 3.7 
cation, with a slight shift from general to Other 2.5 3.7 
special education in the Post Survey. [llllllllliijeHtyRatghiiliittt ll 
Over two thirds of both groups had more General education 50.0 44.0 
than 5 years of teaching experience. The Special education 40.0 48.0 
main difference between these two Teachan equal proportion of both 10.0 8.0 
groups of respondents, as shown in Table Years of teachingexperience* 
10, was the length of time they had spent ‘ess than 1 year a0 £0 
coteaching with a SETIS. There were only 1'°> years god, 238) 
small variances in the 1 to 2 years, 2 to 4 ere aa yeals ee ale 
A 11 to 15 years 7.5 0.0 

years, more than 4 years and not appli- 
: More than 15 years 37.5 40.0 

cable answer choices. However, there 
. Never taught 0.0 4.0 
were notable percent changes in Ue re iilicargth(et aime Goteachingl With SETIS" NNN 
maining two answer choices. Among the | 4<< than 1 year 10.0 36.0 
Pre Survey teacher colleagues, 10.0% S€- 1 to 2 years 175 200 
lected the less than one year choice. That 9 to 4 years 150 16.0 
number rose to 36.0% among the Post wore than 4 years 75 40 
Survey respondents. For the we've never We've never cotaught together before 42.5 16.0 
cotaught together before option, the per- Not Applicable 7.5 8.0 


centage went from 42.5% in the Pre Sur-  *Missingn=2 

vey to 16.0% in the Post Survey. The 

increase of teachers stating they have less than one year of SETIS coteaching experience 
combined with a marked decrease in the percentage of teacher colleagues who had zero 
SETIS coteaching experience may indicate that the teachers are increasing their use and lev- 
erage of SETISs in their schools (see EQ4). 
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Technology use 


As introduced in the Measures and Covariates section (page 5), the Teacher Pre and 
Post Surveys were predominately dedicated to survey items inquiring about technology use 
among the SETIS coteachers and teacher colleagues. These items were designed to address 
EQ4, In what ways have school administrators and teachers leveraged the TIS and the re- 
sources provided by the TIS? Each item asks about the frequency with which teachers em- 
ploy various teaching strategies, categorized into the following indices: Facilitate, Design, 
Model, Promote and Engage. The survey used the following 4-point response scale for all 
items: 1 (Not usually), 2 (Sometimes), 3 (Often), 4 (Almost always). We will begin with an 
examination of the Pre and Post Survey mean scores for the individual items in the five indi- 
ces, followed by a look at statistics for each standardized technology use index. 


Designed to capture the level of technology teacher colleagues use to facilitate and in- 
spire student learning and creativity, the Facilitate index consists of the following survey 
items: 


Facilitate 1. I use information from digital sources to promote learning and engage stu- 
dents in classroom activities. 


Facilitate 2. I require my students to use digital resources and tools for writing, collabo- 
ration, reflection, research, and other assignments. 


Facilitate 3. I engage my students in real-world issues and authentic problem solving. 


Facilitate 4. I require my students to gather information from sources other than their 
textbooks in order to complete their daily assignments (e.g., podcasts, vide- 
os, etc.). 


Facilitate 5. I require my students to present information and actively teach content to 
their fellow students and/or community members. 


Figure 8, shows higher Teacher Post 
Survey mean scores for all items 
compared with mean scores on the 
Pre Survey. The item that showed 
™ Pre Survey &® Post Survey the largest increase, from 2.03 in the 
Pre Survey to 2.46 in the Post Sur- 
vey, was Facilitate 2. However, t 
tests of the Pre and Post Survey Fa- 
| : I Tt cilitate items yielded p values that 
TT were not at the significant level of 
1 7 3 | ‘ 5 0.05 or lower (see Table 26, page 91 
in Appendix E). 


Figure 8. | Comparison of Teacher Pre Survey and Post 
Survey Mean Scale Scores for Facilitation of 
Student Learning and Creativity by Item 
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Pre Survey n = 40; Post Survey n = 26 
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The next index, Design, comprised the following five survey items: 


Design1. I use a blend of both face-to-face and online environments to deliver instruc- 
tion to my students. 


Design 2. I design and/or implement projects that emphasize creative thinking and re- 
quire students to engage in problem solving, decision making and experi- 
mental inquiry, using digital resources/tools when appropriate. 


Design 3. I often design and/or utilize student-centered formative and performance- 
based assessments using available digital resources and tools (e.g., WV 
Writes, Acuity, TechSteps, etc.). 


Design 4. I use digital resources and tools to make assignments for students that are 
based upon their individual interests, abilities, and learning needs. 


Design 5. I require my students to set personal learning goals and to self-assess their 
progress toward meeting those goals, using digital resources and tools when 
available and appropriate. 


All Design items, save Design 1, ex- 
hibited increases from the Pre Survey | Figure9. Comparison of Teacher Pre Survey and Post 
mean to the Post Survey mean (Figure Survey Mean Scale Scores for Design of 

9). The largest increase was seen in Digital Learning Experiences by Item 
Design 2, rising from a Pre Survey 
mean of 2.18 to a mean of 2.52 at Post 
Survey. T tests revealed no significant 4.0 
differences between Pre and Post Sur- 
vey means as the p values, all above 
0.05, ranged from 0.13 to 0.85 (see 
Table 27, page 92 in Appendix E). 


@ Pre Survey Post Survey 


3.0 - 


"REED 
1.0 T T T T 1 
Model, the index intended to 1 2 3 4 5 


measure teacher colleague modeling Item 
of digital-age work and learning tech- | pre survey n = 40; Post Survey n = 25 
niques/behaviors, was created using 
the following four survey items: 
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Model1. I use digital resources and tools to communicate with students. 
Model 2. I use digital resources and tools to communicate with my peers. 


Model 3. I use digital resources and tools to communicate with parents and the com- 
munity outside of my school. 


Model 4. I customize the available digital resources and tools such as WV Writes, Acui- 
ty, TechSteps, etc. to personalize learning for my students. 
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Visible in Figure 10, three of the four 
survey items had higher mean scores in 
the Teacher Post Survey compared with 
the Pre Survey. While none of the items 
presented significant differences when m Pre Survey Post Survey 
t tests were performed (the p values for 
these items ranged from 0.13 to 0.84), 
the item with the largest increase in 
mean score was Model 2 (Table 28, 
page 92 in Appendix E). 


Figure 10. Comparison of Teacher Pre Survey and Post 
Survey Mean Scale Scores for Modeling 
Digital-Age Work and Learning by Item 
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The fourth index, Promote, as- 
sessed teacher colleague promotion 
and modeling of digital citizenship and 
responsibility. The index consists of | Pre Survey 7 = 40; Post Survey n = 25 
three survey items: 


Item 


Promote 1. I advocate, model and teach my students about safe, legal and ethical use of 
digital information and technology, including respect for copyright, intellec- 
tual property, and the appropriate documentation of sources. 


Promote 2. I offer students opportunities to use digital resources and tools to partici- 
pate in collaborative projects with students of other cultures that address 
current problems, issues, or themes. 


Promote 3. Students in my class model appropriate online behavior and social interac- 
tion through digital activities in my classroom. 


All three Promote items yielded in- 


. Figure 11. Comparison of Teacher Pre Survey and Post 
creases in mean scores from the 


Survey Mean Scale Scores for Promoting and 


Teacher Pre Survey to the Post Survey Modeling Digital Citizenship by Item 
(Figure 11). The first item (Promote 1) 
showed the largest increase, growing m™Pre Survey S Post Survey 
from a mean of 2.48 to 2.84. The p 40 
value for this item, along with those of 

3.0 


the other two items, still exceeded 
0.05, meaning none of the items were 
significantly different from Pre to Post 
Survey (Table 29, page 93 in Appendix 
E). 
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Item 
The final index, Engage, in- | Pre Survey n= 40; Post Survey n = 25 
cludes four survey items measuring 


professional development, growth, and leadership activities among teacher colleagues. The 
Engage survey items are: 
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Engage 1. I participate actively in local communities of practice with my fellow teachers, 
either online or face-to-face. 


Engage 2. I participate actively in online/global communities with other educators out- 
side of my local community to gather and discuss resources and ideas related 
to student achievement and learning. 


Engage 3. I model and teach other educators to use digital tools and resources to pro- 
mote student achievement and learning. 


Engage 4. I regularly seek out digital resources, tools, and research and evaluate its 
quality and relevance prior to using it in the classroom. 


Individual items in the Engage 
Figure 12. Comparison of Teacher Pre Survey and Post Survey index evidenced little change 
Mean Scale Scores for Engagement in Professional 


: from the Teacher Pre Survey to 
Development and Leadership by Item 


the Post Survey. Engage 2 

™ Pre Survey © Post Survey showed a minor increase rising 
40 from a mean of 1.75 to 1.79. The 
remaining three items exhibited 
nominal decreases in mean 


3.0 
2.0 - scores (see Figure 12). With 
A i such minor changes, none of the 
1.0 T T T T 
1 2 3 4 


w 
— 
°o 
YY 
2) 

a 
© 
=] 
ww 
c 
© 
wo 


differences from Pre Survey to 
Post Survey were statistically 
significant (see Table 30, page 
93 in Appendix E). 


Item 


Pre Survey n = 40; Post Survey n = 24 


Examined as individual items, there were no significant differences between the 
Teacher Pre Survey means and the Post Survey means; not one individual item’s p value met 
the criteria of 0.05 or less. However, when the items are combined together to create their 
respective standardized technology use indices, the results showed some statistically signifi- 
cant change. 


Figure 13 illustrates Teacher Pre Survey and Post Survey mean scores for each of the 
technology use indices. All of the indices except Engage demonstrated an increased mean 
when comparing the Pre and Post surveys; however statistical tests using a significance level 
of p = 0.05 indicate that only the Facilitate (p = 0.0004) and Design (p = 0.0091) indices 
had large enough increases to be statistically significant (Table 11). The Promote index ap- 
proached statistical significance with a p score of 0.06. 


The West Virginia Special Education Technology Integration Specialist (SETIS) Program | 27 


Results 


Figure 


13. Comparison of Teacher Pre Survey and Post Survey Mean 
Scale Scores by Standardized Technology Use Scales 
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Pre Survey n = 40; Post Survey n = 24-26 


Table 11. Differences in Standardized Technology Use Indices: Pre and Post Means, T Test and Effect Size 
Pre (2011) Post (2012) Significance of difference 

Index Mean SD Mean SD t df p  Cohen’sd_ Effect size 
Facilitate 2.27 0.95 2.60 0.91 3.59 368 0.00 0.36 small 
Design 2.14 0.96 2.32 0.88 2.57 363 0.01 0.20 small 
Model 2.38 1.01 2.49 1.07 0.92 289 0.36 NA NA 
Promote 2.28 1.09 2.52 1.06 1.93 216 0.06 NA NA 
Engage 2.10 0.94 2.04 0.92 0.49 286 0.63 NA NA 


Using Cronbach’s alpha to test internal consistency among items on each given in- 
dex, Table 12 shows the Teacher Pre and Post Surveys’ reliability coefficients (alphas) for the 
five indices. Remembering that an alpha of 0.70 and higher is considered reliable, nearly all 
of the indices presented high to very high reliability. Only one index yielded an alpha below 
0.70. At 0.65, the Pre Survey Promote index may be reliable, but should be interpreted with 


caution. 


Table 12. Technology Use Indices: Reliability Coefficients 

Index Items included Pre Survey a Post Survey a 
Facilitate Survey Section I: Facilitate 1-5 0.84 0.89 
Design Survey Section Il: Design 1-5 0.83 0.78 
Model Survey Section III: Model 1-4 0.82 0.76 
Promote Survey Section VI: Promote 1-3 0.65 0.74 
Engage Survey Section V: Engage 1-4 0.74 0.74 
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Effect sizes 


Only two of the technology use indices, Design and Facilitate, showed statistically 
significant differences between the Teacher 
Pre Survey and the Post Survey results, mak- 
ing them eligible for determining effect size. 
Effect sizes for both indices (i.e., 0.2 and 0.36 
respectively) were small (Figure 14). 


Figure 14. Teacher Pre Survey and Post Survey 
Effect Size by Standardized 
Technology Use Scales 


Design 


. Open-ended items 
Facilitate 


Teacher colleagues were asked to re- 
spond to three essay style open-ended ques- 
tions to provide qualitative data for this 
study. These questions offered teacher col- 
leagues the opportunity to use their own words to describe the role of a SETIS as well as 
what they hoped to accomplish working with a SETIS (Teacher Pre Survey) and what they 
actually accomplished working with a SETIS (Teacher Post Survey). The final open-ended 
question allowed teacher colleagues to provide any additional comments they had about the 
SETIS program. 


0.5 
Effect size 


The first open-ended question on the Teacher Pre and Post Surveys asked teacher 
colleagues to describe their perception of the role of a SETIS. In the Pre Survey, 29 of the 40 
respondents answered this question; in the Post Survey 18 of 27 teacher colleagues respond- 
ed to the question. The responses were coded into six broad themes, with one theme divided 
into five subthemes (see Table 13). In both the Pre and Post Surveys, two themes were domi- 
nant: technology resource-use person for teachers, and incorporate/integrate technology- 
use in the classroom. 


The theme technology resource-use person for teachers was mentioned by 11 teacher 
colleagues in the Pre Survey and eight in the Post. The following are examples from the Pre 
Survey: 


To help with tech. questions and problems, to keep us informed of available resources 
and new ideas that can help us do our jobs better. 


To provide support and insight into technology that is available for teachers to use 
with students. 


To assist and encourage me as a teacher to move forward with the use of technology 
in my classroom because I feel uncomfortable in this area of education. 


Examples from the Post Survey include the following: 


To provide technical assistance to teachers when needed. To share new and exciting 
websites and online activities for our staff and students. 


To work with and assist the classroom teacher in meeting special ed. student needs 
with the help of technology. 


In my opinion the role of the TIS is to support the education of students by training 
the classroom teacher on the latest technology, assisting the teachers in finding and 
utilizing resources to aid in connecting technology to CSO's [Content Standards and 
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Table 13. Teacher Perception of SETIS Role Objectives], and to provide educa- 


; tional experiences for the students 
Frequency of comments* that will make learning exciting 
Pre Survey Post Survey and relevant. 


EUs a2) us te) The other dominant theme was 
Technology-use resource person: ‘ , 
incorporate/integrate technology-use 
for teachers 11 8. 
in the classroom. Teacher colleagues 
for students 1 dl : ; ‘ : 
responded with this theme 13 times in 
for teachers and students 7 3 : 
the Pre Survey and seven in the Post 
for entire school 1 0 : 
nm eenerel 2 1 Survey. Some Pre Survey examples in- 
Facilitator of students’ learning 2 2 clude the following: 
experience I think the role of the SETIS is to 
Incorporate/integrate technology 13 7 effectively integrate a variety of 
use in the classroom technology resources and tools into 
A coteacher al 2  theclassroom. 
Important to school 1 1 To provide ideas on incorporating 
Not sure 3 0 technology into the classroom, not 
*Some respondents provided more than one comment, therefore, only for the teacher, but also for 
the frequency of comments may exceed the number of the students. 


respondents (n). 


I feel the role of the SETIS is to 
help teachers incorporate different medias and ideas into their lessons in order to en- 
gage the students they are working with as well as help create ideas for the teachers to 
use in all their classes. 


To help me integrate digital resources into my classroom more on a daily basis. 
Post Survey examples of this theme include the following: 


Our SETIS is available to help us implement technology into our existing lessons as 
well as collaborate with us to create new lessons and units based on technology and 
problem-solving skills. 


The role of the SETIS is to help teachers design lessons that incorporate technology 
and can be differentiated to meet the needs of all students. The SETIS also works with 
students as they integrate technology into their personal learning. 


To support me in adding opportunities for students and myself to integrate technolo- 

gy into every aspect of education. 

One theme that was repeated often in the Teacher Pre Survey, but not in the Post 
Survey, was technology-use resource person for teachers and students. In the Pre Survey 
this theme was mentioned seven times while it was sited three times in the Post Survey. No 
cause can be attributed to this shift since we do not know if the same teacher colleagues re- 
sponded to this question in both surveys. It may reflect a change in the perceived SETIS role, 
or it may be a manifestation of differing opinions from different people. Most of the other 
themes remained consistent from Pre to Post Survey, save one. The not sure theme was 
found three times in the Pre Survey and not at all in the Post Survey. 


The second open-ended question varied slightly from the Teacher Pre Survey to the 
Post Survey. In the Pre Survey it asked teacher colleagues what they hoped to accomplish by 
working with a SETIS throughout the school year. The Post Survey question asked what they 
actually accomplished by working with a SETIS. Respectively, there were 28 Pre Survey and 
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18 Post Survey respondents to this 


Table 14. Teacher Expected and Observed Outcomes From 
Working With a SETIS question. Eight themes were iden- 
Frequency ot ecrment® tified, including not sure and N/A 
Pre Survey Post Survey categories (Table 14). 
Theme (n = 28) (n = 18) Three major themes emer-ged: 
Increased technology knowledge 7 6 (a) increased technology 
Increased technology support for 1 0 knowledge, (b) increased technol- 
srucens ogy use in the classroom, and (c) 
cael oe a : enhanced student experience. The 
Use of specific technology tools (WV 2 5 leading theme in the Pre Survey 
Writes, Tech Steps, Acuity, etc.) was increased technology use in 
Improved student performance 3 2 the classroom, with 12 observa- 
Enhanced student experience 6 4 tions. This theme remained strong 
Not sure 1 0 inthe Post Survey, with six teacher 
N/A 1 1 colleagues identifying it as an ac- 
*Some respondents provided more than one comment, therefore, the complishment resulting from 
frequency of comments may exceed the number of respondents (n). working with a SETIS. 


Some Pre Survey examples include the following: 


I hope to accomplish a more blended classroom environment by using face to face 
and also online learning. I hope to see technology as a part of my daily lessons and in- 
crease student engagement. With the help of the SETIS I hope to see a spike in criti- 
cal thinking in our students and quality projects. 


To become more comfortable with using the tech. available in my classroom and use 
it more productively with my students on a daily basis. 


I hope to increase the amount of technology and 21st Century skills my students use. 


I hope to learn how to incorporate different technologies and medias into my class- 
room to help engage and teach my students. 


Integration of digital resources with my students at levels that are appropriate for 
them. 


Post Survey examples of this theme include the following: 


I accomplished a greater confidence in incorporating technology into my lessons. My 
students, however, accomplished so much more because of the skills of our SETIS. 


I learned more about online resources such as online interactives (Twiddla) and how 
to build a website. Collaborize classroom also became an integral part of our class- 
room (and beyond) learning. 


They provided me with virtual field trips I could use with the preschoolers I teach. 


The second theme that remained dominant, despite fewer teacher colleagues re- 
sponding to the question, was increased technology knowledge with seven observations in 


the Pre Survey and six in the Post Survey. 


Some Pre Survey examples include the following: 


To further extend my knowledge base of best practices by incorporating technology as 
much as possible into my daily instruction. 
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I hope to learn more about integrating technology and become more familiar and 
comfortable with it. 


To learn more about what is available to me and my class using technology. 
Post Survey examples of this theme include the following: 


I learned many new tools and how to use them. 


I accomplished learning how to use several (new to me) technology tools to aid in im- 
plementing the required CSO's [Content Standards and Objectives]. Our SETIS is 
continuously finding and sharing new ways to use the technology that is available at 
our school. 


I learned how to provide blended instruction for student's with visual impairments 

using face-to-face contact and online interactions. I am continuing the online interac- 

tions even though I am no longer working at that school. I also learned firsthand 

about some additional technology tools to support students with [Title] VI. With The 

SETIS at my regular job placement I learned about potential resources and received 
support dealing with technological glitches. 

The third main theme, enhanced student experience, produced six observations in 


the Pre Survey and four in the Post Survey. 
Some Pre Survey examples include the following: 


I hope to find ways to open technology to my students to enhance their educational 
experience. 


I hope to give our SE students more opportunities relevant to their learning level. 
To make my classroom the best learning environment as possible for my students. 
Post Survey examples of this theme include the following: 


My students were able to have more direct instruction. 


We created many problem solving, student-centered, and technology enhanced les- 

sons and units that provided engaging learning for our students. 

There were also several comments, mostly in the Post Survey, on the use of specific 
technology tools. While not a certainty, it is reasonable to attribute this increase to the rise in 
teacher colleague knowledge and awareness of technology resources available through inter- 
action with a SETIS. 


The final open-ended item on both the Teacher Pre and Post Surveys was one that al- 
lowed teacher colleagues to provide any additional comments they had about the SETIS pro- 
gram. In the Pre Survey, 13 teachers wrote an additional comment; in the Post Survey four 
teachers responded to the item. Seen in Table 15, six overarching themes were discovered in 
these comments. Results from the Pre Survey indicate mainly positive comments concerning 
the SETIS program, followed by comments centered on benefits teachers and students re- 
ceive from the SETIS and concerns about the availability of technology within classrooms. 
As you will see from the Pre Survey samples below, some comments discuss more than one 
theme: 


I am excited for a new year with the SETIS program. I look forward to the new ways 
of learning and teaching. 


Good idea as long as the technology is up to date and working properly. 
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I think this program is a great asset to our school and the group of kids that we teach. 
I hope that the knowledge gained will accompany funds to have new technologies ac- 
tually in all classrooms. 


This program has great promise for both educators and students because of the con- 
tent it can deliver and because of the collaborative example it sets for the students. 


Because there were only four responses to the additional comments item on the Post 
Survey, it was not possible to compare the Pre Survey results to the Post Survey results. 
However, all of the comments were complementary of the SETISs, indicating satisfaction 


and successful relationships. Here 
Table 15. Teacher Colleague Additional Comments are two examples from the Post 


Frequency of comments* Survey comments: 


Eisley Rost uey The TIS is a big piece in my 


Theme (n = 13) =) network. She gives me the 
Complementary of SETIS 2 4 support I need to venture into 
Complementary of SETIS program technology when I would not 


Teachers benefit have otherwise. 


Teachers and students benefit I am grateful to be co-teaching 
Concerned about availability of with my SETIS. She brings new 
technology in classroom ways of looking at the curricu- 
g lum to my classroom and she is 


a wonderful resource for other 


wWwwou 
Orr O 


Not sure 1 


*Some respondents provided more than one comment, therefore, the 


frequency of comments may exceed the number of respondents (n). classes that I teach as well. 


Administrator Pre and Post Surveys 


School administrator demographics 


There were a total of nine respondents to the Special Education Technology Integra- 
tion Specialist (SETIS) Administrator Pre Survey (hereafter, Administrator Pre Survey), and 
13 to the Special Education Technology Integration Specialist (SETIS) Administrator Post 
Survey (hereafter, Administrator Post Survey; see Table 16). However, as with the Teacher 
Pre and Post Surveys, only counties that responded to both the Administrator Pre Survey 
and Post Survey were included in the data analysis. After excluding counties that did not 
have representation in both the Pre and Post Surveys, there were eight Pre and Post Survey 
responses included in the analysis. 


School administrators were asked to identify the role they held within their school. 
The options were principal, assistant principal, and other. Illustrated in Table 17, the ma- 
jority of respondents were school principals; representing 62.5% in the Administrator Pre 
Survey, and 87.5% in the Post Survey. The remaining respondents were all assistant princi- 
pals, at 37.5% in the Pre Survey and 12.5% in the Post Survey. No respondents chose the oth- 
er category. Comparing the roles between the Pre and Post Surveys, it is clear there was a 
higher percentage of assistant principals participating in the Pre Survey than the Post Sur- 
vey. Conversely, a higher percentage of school principals participated in the Post Survey 
than the Pre Survey. 
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The Pre Survey also included an item 
about awareness of SETIS presence within the 
administrator’s school. They were asked to re- 
spond yes or no to the following statement, 
“Before receiving this survey, I was aware that 
my school would have a SETIS for the 2011-12 
academic year.” Six out of the eight adminis- 
trators (75.0%) confirmed they were, indeed, 
aware their school would have a SETIS 
throughout the school year. 


Likewise, the Post Survey contained an 
item not included in the Pre Survey. This item 
asked administrators, “Based on your class- 
room observations this school year, has the 
SETIS candidate increased student engage- 
ment in curricular activities through the in- 
creased use of technology?” The responses 
were overwhelmingly positive (Figure 15). Of 


Table 16. 


County Location of Administrator 
Respondents' School 


Provider 


Pre Post 
(n =9) (n = 13) 


Total 


13 


Boone 
Greenbrier 
Hampshire 
Harrison* 
Marion* 
Mineral 
Mingo* 
Putnam 
Raleigh 
Randolph 
Tucker* 
Upshur* 


PPNRPOROORP BIO 


PORPNRPRRBRPNBRP PB 


0 


*Counties excluded in data analysis due to lack of 
representation in either the Pre Survey or the Post 


Survey. 


the eight respondents, six gave this question the highest possible rating (5) and the remain- 


ing two chose the second highest rating (4).5 


Use of SETISs 


The Administrator Pre and Post Sur- 
veys were focused primarily on answering EQ4: 
In what ways have school administrators and 
teachers leveraged the TIS and the resources 
provided by the TIS? These surveys included 
fewer items than the Teacher Pre and Post Sur- 
veys, and were constructed differently 
than the other surveys in the study 
(see Appendix C, page 77 for the full 
version of surveys). Beyond the demo- 
graphic information discussed in the 
previous section, the Administrator 
Pre Survey consists of six multiple- 


oy 
an 
< 
i=} 
[= 
7) 
) 
4 


choice statements and three open- (Untrue) 1 


ended items. While the Post Survey 
also contains three open-ended items, 
it has one additional multiple-choice 
item (seven total). 


Figure 15. 


Table 17. Role of Administrator Respondents in 
Their Schools 

Pre percent Post percent 

Role (n =8) (n= 8) 

Principal 62.5 87.5 

Assistant principal 37.5 12.5 

Other 0 0 


Administrator Post Survey: Increased 


Student Engagement via Use of Technology 
(rue) 5 
B=] 


4 


3 
2 


Frequency 


5 Note: The administrators who responded to the Pre Survey are not necessarily the same as 


those responding to the Post Survey. 
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The Administrator Pre Survey multiple-choice items are phrased to ask administra- 
tors how they intend to use their SETIS, such as, “I plan to ask my SETIS to model the inte- 
gration of technology for their coteachers and others within the school.” The Post Survey, on 
the other hand, asks school administrators to report how they actually utilized their SETIS 
during the course of the school year. Looking at the same example, the multiple-choice 
statement becomes, “My SETIS modeled the integration of technology for his/her coteachers 
and others within the school.” Further, both the Pre and Post Surveys utilize a 5-point Lik- 
ert-type scale. However, the indices differ slightly. The Pre Survey scale is a rating of likeli- 
hood, with the lowest option, 1 (unlikely) and the highest, 5 (likely). The Post Survey also 
includes a response range from 1 as the lowest possible choice to 5 as the highest. It is a scale 
of agreement, however, ranging from complete lack of agreement, indicated by selecting 1 
(untrue), to complete agreement, indicated by selecting 5 (true). Pre and Post Survey items 
are listed in Table 18. 


Table 18. Administrator Pre and Post SETIS Use Survey Items 


Item Pre Survey Post Survey Statement 

Lead professional | plan to ask my SETIS My SETIS shared... ... what he/she has learned by leading 

development to share... professional development for the other 
teachers in my school 

Model technology | plan to ask my SETIS My SETIS modeled... ...the integration of technology for 

integration to model... his/her coteachers and others within the 
school 

Contribute to | anticipate asking my My SETIS assisted... ...mein developing the school’s strategic 

strategic plan SETIS to assist. . . plan with regard to information and 
technology needs 

Customize digital | expect that my SETIS My SETIS assisted... ... his/her coteachers in customizing 

resources will assist. . . available digital resources and tools such as 


WV Writes, TechSteps, Acuity, Edmodo, 
Thinkfinity, etc. to personalize learning for 


students 
Identify digital | will request that the Irequested thatthe  ... work with teachers to identify digital 
resources SETIS. . . SETIS. . . resources and tools that effectively 
integrate technology into their current 
curriculum 
Analyze student —_| will ask the TIS... lasked the SETIS... —... to conduct analyses of student data and 
data engage in action research to help me 


understand the impact of technology 
integration at my school 


After exclusion for counties that did not have representation in both the Administra- 
tor Pre and Post Surveys, eight survey responses remained for analysis. While eight survey 
responses provide a small cohort and results must be interpreted with caution, the partici- 
pating school administrators completed all multiple choice survey items. 
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Illustrated in Figure 16, results indicated school administrators’ intention to utilize 


their SETISs was high. Among the Pre Survey mean s 


cores, the lowest average was 4.0 (ana- 


lyze student data) while the highest was 4.6 (identify digital resources). Still, despite their 
high level of intent in the Pre Survey, all Post Survey items save one (model technology inte- 
gration) showed slight decreases in their mean scores. The differences though, even for the 
one item that showed an increase in survey mean, were not statistically significant (see Table 


31, page 95 in Appendix F). 


Figure 16. School Administrator Expected and Actual SETIS Use by Survey Item 
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Item 


Open-ended items 


The essay style, open-ended items on the Administrator Pre and Post Surveys are 
congruent with the open-ended questions on the Teacher Surveys. The first item asked 


Table 19. Administrator Perception of SETIS Role 


school administrators to describe 
the role of a SETIS. As shown in Ta- 


Frequency of comments* 


ble 19, most administrators expected 


Pre Survey Post Survey 


SETISs to take the role of a resource 
person for teachers. This trend was 


Theme (n =8) (n =8) 
Technology use resource person for 6 8 
teachers 

Technology use resource person for 2 0 


teachers and students 


consistent across the Pre and Post 
Surveys suggesting the role for 
SETIS that administrators expected, 


*Some respondents provided more than one comment, therefore, the 
frequency of comments may exceed the number of respondents (n). 


proved to be their actual role 
throughout the school year. Selected 


examples are listed below. 


The following are examples from the Pre Survey: 


To help teachers integrate technology in their classrooms. This includes helping them 


find resources, operate equipment, and trainings. 


I believe the role of the SETIS is to assist coworkers with the technology and help to 
eliminate the hesitance of integrating it into the lessons. She should provide sugges- 
tions, answer questions, and be willing to model lessons for her coworkers. 
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Examples from the Post Survey include the following: 


Serve as a technological resource for professional development, digital resources and 
tech tools and to assist in the implementation of such into the curriculum and in- 
structional practices of our teachers. 


To be an effective resource for the teachers to be able to collaborate with in order to 
improve their technology skills. 


Results 


The second open-ended question asked school administrators to describe the out- 
comes they expected to see (Pre Survey) and the outcomes they actually observed (Post Sur- 


vey) as a result of having a SETIS 
in their school. Seven administra- 
tors responded to this question in 
the Pre Survey, and eight in the 
Post Survey. Pre Survey responses 
were varied, with administrators 
citing increased technology use, 
improved student performance, 
and staff development/technology 
integration as their leading expec- 
tations (Table 20). However, 
comments shifted focus in the Post 
Survey with most administrators 


Table 20. Administrator Expected and Observed Outcomes 
From Having a SETIS 


Frequency of comments* 


Pre Survey Post Survey 


Theme (n =7) (n= 8) 
Increased technology use 3 6 
Provide technology support 1 1 
Improved student performance 4 1 
Staff development — technology 3 2 
integration 


*Some respondents provided more than one comment, therefore, the 
frequency of comments may exceed the number of respondents (n). 


naming increased technology use as the outcome they observed in their schools as a result of 
having a SETIS. The following are examples from the Pre Survey: 


Technological use will increase amongst teachers and students. Integration of tech- 
nology to assist with improving student achievement. Technology will become inte- 
grated into our teacher’s instructional practices and our students learning behaviors. 
(Increased technology use and Staff development/technology integration) 


I want our school to integrate technology and use it as a tool. I do not want it to be 
viewed as an extra something to do in the classroom. I hope that the SETIS can help 
to accomplish this goal, but also supply suggestions and help to her co-workers to as- 
sist our at risk students with technology interventions to keep them interested and 
successful. (Improved student outcomes and Staff development/technology integra- 
tion) 


Examples from the Post Survey include the following: 


We were able to set-up and incorporate i-pod lab into our reading language arts clas- 
ses and anticipate the integration into all classes for next year. Greater differentiation 
of instruction using technology as a tool. Professional development for all staff with 
tech tools (i-pods) and digital resources (plato learning, acuity,edmodo, etc.). (In- 
creased technology use and Staff development/technology integration) 


I noticed that more teachers started using technology and went to the TIS for needed 
support in the area of technology. (Increased technology use) 


Few school administrators provided additional comments in either the Pre or the 


Post Survey. Of the four Pre Survey respondents, most were complementary of the SETIS or 
the SETIS program. One school administrator voiced concern about funding sources for the 
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SETIS. Only one administrator offered an additional comment in the Post Survey; this com- 
ment was very complementary of the work the SETIS had accomplished in their school. 


There were not enough responses to provide an adequate number of samples; the results are 
outlined in Table 21. 


Table 21. Administrator Additional Comments 


Frequency of comments* 


Pre Survey Post Survey 


Theme (n = 4) (n = 1) 
Complementary of SETIS 2 1 
Complementary of SETIS program 1 0 
Monetary concerns 1 0 


*Some respondents provided more than one comment, therefore, the 
frequency of comments may exceed the number of respondents (n). 
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This study examines survey results from the three primary stakeholder groups in the 
SETIS program: SETISs, teacher colleagues, and school administrators. These survey results 
enable us to address each of the evaluation questions chosen to measure the implementa- 
tion, use, and impact of the SETIS program. 


Evaluation Question 1 


The first evaluation question—To what extent does the TIS program build the capac- 
ity of participating TISs to plan and facilitate: (a) teaching and learning, (b) information 
access and delivery, and (c) program administration?—is addressed through the Special 
Education Technology Integration Specialist (SETIS) Survey (hereafter, SETIS Survey) ca- 
pacity indices. 


The organizational and human capacity indices of the SETIS Survey provide evidence 
of the increased capacity of SETIS to plan and facilitate teaching and learning. Organiza- 
tional capacity survey items assessed the ability of SETISs to interact, communicate, and col- 
laborate with others (i.e. teach). All items from this index demonstrated statistically 
significant increases in SETISs’ retrospective postprogram means compared with their pre- 
program means, and very large effect sizes. Of particular note, the item yielding the largest 
increase was, “I collaborate with others within the school to effectively integrate technology 
into instruction,” indicating increased capacity among SETIS to teach teachers how to opti- 
mize technology use in their classrooms. The human capacity index provided support for 
gained capacity among SETISs to plan and facilitate learning. Of all the indices, this index 
showed the greatest increase in retrospective pre-to-post means, as well as the largest effect 
size. Additionally, all 17 of the individual items comprising this index showed significant in- 
creases in mean scores and very large effect sizes. The largest gains were seen in the human 
capacity items, with eight of the 17 items yielding mean scores that rose by 2+ points on a 5- 
point scale in respondents’ retrospective pre to post self-assessments of their capacity. Fur- 
ther evidence of enhanced learning among SETISs can be seen in the results from the open- 
ended survey item, “What would you consider to be the most valuable aspects of participat- 
ing in the TIS program.” Nine of the 11 respondents provided answers that fell within the 
theme, learning about technology resources/tools. 


The SETIS Survey’s material capacity and organizational capacity indices focused on 
the second part of EQi—that is, measuring perceived increases in SETISs’ capacity to plan 
and facilitate information access and delivery. Resource access, development, and sharing 
were each covered in the material capacity index. All five survey items produced statistically 
significant differences in survey means (three of five items showed an increase in mean score 
greater than 2 points on a 5-point scale) along with large or very large effect sizes. Further, 
the survey items in the organizational capacity index provide additional validation that 
SETISs experienced enhanced capacity in delivering technology integration to their schools. 
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The structural capacity index examined the final component to EQ1, SETISs’ capacity 
to plan and facilitate program administration. In general terms, structural capacity denotes 
elements such as policies, procedures, and formalized practices. For the purposes of this 
survey, the items making up the structural capacity index concentrated on policies and prac- 
tices SETISs observed at their schools and with their teacher colleagues. While SETISs do 
not directly change policies, procedures, and formalized practices at their schools, positive 
impact on these elements may be evident if SETISs are capable of successful program ad- 
ministration. As discussed in the results section, while increases in mean scores on retro- 
spective pre- to postprogram items were not as dramatic, all items in this index remained 
statistically significant. Effect sizes ranged from medium to very large. These results indicate 
SETISs are having a positive and meaningful impact on structures within the schools they 
support. 


Overall, the 2011-2012 SETIS cohort perceived substantial increases in their capacity 
to plan and facilitate all of the components in EQ1. Results from the SETIS Survey produced 
more statistically significant differences in mean scores as well as larger effect sizes than ei- 
ther of the other two surveys deployed in this evaluation study. The strength of these results 
provide reasonable evidence that the 2011-2012 SETIS program equipped its candidates 
with the capacity necessary to implement technology integration techniques within their 
schools and classrooms. 


Evaluation Question 2 


To answer the second evaluation question—To what extent do TISs encounter barri- 
ers to successful program implementation (e.g., financial, temporal, relational, etc.)?—we 
will use findings from the SETIS Survey and the SETIS Administrator Survey. From the 
SETIS Survey, three items pertained to this question. One item comes from the material ca- 
pacity index (Item 40) asking SETTSs if they have access to high quality technology resources 
(e.g. computers, digital cameras, whiteboards) at their schools. The retrospective prepro- 
gram mean for this item was 3.85 on the following 5-point scale: 1 (strongly disagree), 2 
(disagree), 3 (neutral), 4 (agree), and 5 (strongly agree). The postprogram mean was 4.54. 
While this result is a significant increase, it is not as dramatic as increases seen for other ma- 
terial capacity items; it is important to note that the preprogram mean for this item was con- 
siderably higher than the preprogram means for the other items in the index. Accordingly, 
this item generated a large effect size while the other items in the same index returned very 
large effect sizes. These findings indicate access to high quality technology resources at 
schools was not seen as a barrier before the SETIS program, and was seen as even less of a 
barrier after the program. 


The remaining items drawn from the SETIS Survey to address the question about 
barriers were open-ended questions. Results from the first question—“What would you 
change about the TIS program if you could?”—revealed four major themes (plus a N/A cate- 
gory for one response). Of the nine responses, four cited a desire to see program and/or con- 
tent change. These recommendations included more stringent accountability, increased 
guidance, and changes to specific course assignments. Three respondents suggested meeting 
changes; specifically, SETISs want more face-to-face and SETIS-only meetings. Two re- 
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sponses were centered on time changes with one wanting more time and the other suggest- 
ing less time to complete assignments. Two SETISs stated that they did not wish to see any 
changes in the program. While the responses to this question vary considerably, it would 
seem most SETIS candidates would like a program with amplified accountability and added 
SETIS-only in-person meetings. 


The second open-ended question, and most informative to answering EQ2 was, 
“What barriers, if any, did you encounter in implementing what you learned as part of the 
TIS program?” Of the 11 respondents, four named time as their greatest barrier. SETISs ex- 
pressed a lack of time in general, a shortage of time spent in a classroom setting, and not 
enough time to implement what they have learned in the program. Three respondents cited 
access to/quality of technology as the primary barrier they encountered, including limited 
internet access and broadband/bandwidth issues; limited access to computer labs and com- 
puter access for students outside of the classroom; and difficulty in acquiring student e-mail 
accounts. A shortage of common planning between teacher colleagues and SETISs was men- 
tioned twice and a lack of collaboration (also between teachers and SETISs) was identified 
once. 


A final barrier worthy of mention was discovered in the Special Education Technolo- 
gy Integration Specialist (SETIS) School Administrator Pre Survey (hereafter, Administrator 
Pre Survey). Administrators were asked if they were aware their school would have a SETIS 
during the 2011-2012 school year; 75% responded they did know a SETIS would be in their 
school. Ideally, 100% of school administrators would be aware of the presence of a SETIS in 
their schools. This finding could indicate a barrier in the communication flow between the 
key stakeholders of this program. 


Generally, SETIS and school administrators in the 2011-2012 SETIS program report- 
ed varied and relatively moderate barriers. From the perspective of the SETISs, a lack of 
time was their largest barrier. Access to technology tools among SETISs was not an issue, 
however, computer access for students and internet speed was a major concern. The SETISs 
also identified several things they would change about the program; while these may or may 
not be barriers to program implementation, they should be taken under consideration. Fi- 
nally, as demonstrated in the Administrator Survey, 25% of the administrators participating 
in the survey were not aware a SETIS would be present in their schools. 


Evaluation Question 3 


Findings from the Special Education Technology Integration Specialist (SETIS) 
Teacher Pre and Post Surveys (hereafter, Teacher Pre and Post Surveys), along with the 
SETIS Administrator Pre and Post Surveys, were used to address EQ3, To what extent is the 
level of technology integration in TIS schools positively impacted through participation in 
the program? The first evidence of positive impact was an increase in the percentage of 
teachers with SETIS coteaching experience. Teacher Pre Survey results revealed only 10% of 
teacher colleagues had less than 1 year of coteaching experience with a SETIS, while more 
than 40% of the teachers in the Pre Survey had never cotaught with a SETIS. The percentage 
of teachers who had less than a year of coteaching experience rose to 36% in the Teacher 
Post Survey by the end of the school year, while the percent with no coteaching experience 
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dropped to 16%—denoting a substantial increase in teacher colleagues working with a SETIS 
for the first time during the 2011-2012 SETIS program. 


Open-ended questions from both the Teacher and Administrator Pre and Post Sur- 
veys proved to be valuable sources of information while measuring the positive impact of 
school participation in the SETIS program. When teacher colleagues were asked what they 
hoped to accomplish working with a SETIS (Teacher Pre Survey) and what they actually ac- 
complished (Teacher Post Survey) the predominant themes were increased use of technolo- 
gy in classrooms (Pre = 42.9%, Post = 33.3%), improved technology knowledge among 
teachers (Pre = 25%, Post = 33.3%), and enhanced student experiences (Pre = 21.4%, Post = 
22.2%). These results imply SETISs were successful in not only building technology 
knowledge among teachers and integrating technology use in classrooms, but also positively 
impacting student experiences. Sustaining this finding, 75% (6/8) school administrators 
stated they observed increased technology use as a result of having a SETIS in their schools. 
Furthermore, from the Teacher Post Survey, of the 4 respondents who wrote additional 
comments, 100% were complimentary of the SETIS individual they worked with throughout 
the school year. 


The technology use indices from the Teacher Survey were designed to help answer 
EQ3 as well. Though the individual items that make up the Facilitate, Design, Model, and 
Promote index items revealed increases in mean scores from the Pre Survey to the Post Sur- 
vey, the results were not large enough to be considered statistically significant. Individual 
items on the Engage index changed very little from Pre to Post means and were far from sig- 
nificant levels. When the individual items were aggregated to create the Technology Use in- 
dices, Facilitate, Design, Model, and Promote indices demonstrated increased mean scores. 
Two indices, Facilitate and Design, produced significant differences from Pre to Post Survey 
means, and Promote yielded a p value that was near significance. Shadowing the individual 
item results, the Engage index underwent little change. The effect sizes were small for the 
two indices that did show significant results (Facilitate and Design). All in all, the results for 
the technology use indices were not large enough to indicate positive or negative changes. 


SETIS activities that led to positive impacts in classrooms were associated with nota- 
ble increases in coteaching experience among teacher colleagues and SETIS, improved tech- 
nology integration in classrooms, raised technology knowledge among teachers, and 
enhanced student experiences. Although the technology use indices from the Teacher Pre 
and Post Surveys did not show dramatic shifts or large effect sizes, most of them revealed 
positive increases in mean scores. 


Evaluation Question 4 


The fourth evaluation question—In what ways have school administrators and 
teachers leveraged the TIS and the resources provided by the TIS?—is answered using 
many of the same findings discussed in EQ3, in addition to items from the SETIS Adminis- 
trator Pre and Post Surveys and open-ended questions from the SETIS Survey. That school 
staff began leveraging SETIS services during the course of the year can be seen in increased 
teacher colleague reports of coteaching experience with SETISs at the end of the school year 
compared to the beginning. Also, as discussed in EQ3, teachers and administrators reported 
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increased technology use in classrooms (with teacher’s also citing improved technology 
knowledge), further supporting leverage of SETIS candidates and resources. Further evi- 
dence of leverage can be detected in survey items focused on expected and actual use of 
SETIS candidates and resources. As considered in the EQ3 discussion, teacher colleagues 
reported similar expected and actual accomplishments resulting from working with a SETIS 
during the 2011-2012 school year. Moreover, school administrators predominantly described 
an increase in technology use (Post = 75%) as the outcome they observed from having a 
SETIS in their school. 


Leverage is also evident in survey items asking teachers and school administrators 
about the role of a SETIS. At the time of the Teacher Pre Survey, teacher colleagues primari- 
ly believed the role of the SETIS was to be a resource for teachers (37.9%) and to integrate 
technology use in classrooms (44.8%). These opinions remained consistent in the Teacher 
Post Survey, with 44.4% of the respondents citing a SETIS’s role as a resource person for 
teachers, and 38.9% stating their role is to integrate technology use into classroom settings. 
Among school administrators’ responses concerning the role of a SETIS also remained con- 
sistent; in the Administrator Pre Survey 75% (6/8) said a SETIS’s role is a technology use 
resource for teachers, in the Post Survey 100% (8/8) of the responding administrators 
voiced the same opinion. These data suggest both expectations and outcomes among school 
administrators and teacher colleagues are being met, providing support that SETISs and the 
resources they provide are being utilized (or leveraged) as planned. 


Two open-ended questions from the SETIS Survey give extra insight into the ways 
school administrators are leveraging their SETIS. The first asks SETIS candidates to de- 
scribe structures in their school that allowed them to work meaningfully with their teacher 
colleagues. Of the 11 responding SETISs, seven (63.6%) identified administrative support 
and four (36.4%) named common planning time as the structures leading to meaningful 
work between teacher colleagues and SETISs. The second question, also with 11 responses, 
asked SETISs to specify strategies they used to work with school administrators to address 
technology related issues and concerns. Even though the responses varied, discuss re- 
sources/technology needs was the most common theme, and only one SETIS reported not 
working with a school administrator at all. Results from these two questions indicate school 
administrators are actively supporting and collaborating with the SETIS in their school. 


Finally, to measure leverage we looked at the multiple choice items from the Admin- 
istrator Pre and Post Surveys. Reviewed in the Results section, administrators reported very 
high levels of intent to leverage their SETISs in the Pre Survey. Despite the decrease in these 
levels for many of the survey items (the decreases were not statistically significant) they re- 
mained high, signifying administrators utilized SETISs more or less as they had anticipated. 


All in all, it appears teacher colleagues and school administrators leveraged SETIS’s 
and the resources they bring to a school in the ways they expected throughout the 2011-2012 
SETIS program. While it was concerning that only 75% of the responding school administra- 
tors were aware they would have a SETIS placed in their school, the SETISs named adminis- 
trative support as the most common factor in facilitating meaningful collaboration between 
teachers and themselves. 
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Evaluation Question 5 


The final evaluation question—What impact has the TIS program had on students’ 
technology literacy in participating schools?—relies upon the premise that successful pro- 
gram implementation along with efficient SETIS and technology resource use leads to higher 
student technology literacy levels. To address EQ5 we draw insight from the Teacher Pre and 
Post Surveys and the Administrator Pre and Post Surveys. 


Qualitative open-ended survey items from both surveys provided the most compel- 
ling evidence that the premise is true. When asked about accomplishments and outcomes 
from participation in the SETIS program, 33.3% (6/18) of the teacher colleagues and 75% 
(6/8) of the school administrators cited increased use of technology in classrooms/schools 
as an accomplishment. Several teachers (four of 18) named enhanced student experiences as 
the most notable accomplishment. Also worth mentioning were the two teachers and one 
school administrator who listed improved student performance. From these open-ended 
remarks, it is evident both teachers and administrators observed not only increased technol- 
ogy use in classrooms, but also enriched student experiences and, in some instances, im- 
proved student outcomes. 


All technology use items (Teacher Pre and Post Surveys), with the exception of the 
Engage index, contain items with reference to students. Facilitate items (2—5) ask teachers 
to gauge how often they have their students work with certain types of technology and tech- 
nology-related learning methods. Each of these items showed an increase in mean score 
from the Pre to Post Survey, although none of the differences were statistically significant. 
Every Design item was centered on technology integration into student instruction and ac- 
tivities. While the individual items were not statistically significant, the index as a whole 
proved to be significantly different from Pre to Post Survey; the effect size for the index was 
small. The Promote index, consisting of three items about appropriate and collaborative 
technology utilization were each centered on students. Although not statistically significant, 
this index was near the level of significance with a p-value of 0.06; the effect size was small. 
Keeping in mind the index for these survey items—1 (not usually), 2 (sometimes), 3 (often), 
and 4 (almost ahvays)—many of the items discussed above had a Pre Survey mean score 
near 2 (sometimes) that rose to a Post Survey mean closer to 3 (often). 


A final measurement of impact on student technology literacy may be pulled from the 
Administrator Post Survey. Administrators were asked, “Based on your classroom observa- 
tions this school year, has the SETIS candidate increased student engagement in curricular 
activities through the increased use of technology?” On a rating scale of 1 to 5, 75% (6/8) of 
the responding administrators answered with the highest possible agreement (5) and the 
other 25% (2/8) chose the next highest agreement rating (4). 


It is evident both teacher colleagues and school administrators perceived sizeable 
growth in technology use in classrooms. Further, administrators were congruent in reporting 
they had observed increased student engagement as a result of integrating technology into 
their classwork. Finally, although the technology use indices from the Teacher Pre and Post 
Surveys did not show dramatic changes, many of the individual item mean scores moved 
from (2) sometimes to (3) often. This shift in behavior indicates that, as a result of the SETIS 
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program, teacher colleagues who responded to the survey are practicing technology integra- 
tion techniques and engaging their students using technology resources provided by their 
SETIS. 


Limitations of the Study 


When interpreting results in any study, it is important to consider inherent limita- 
tions that may skew findings. While sound research methods are an integral component of 
high quality studies, eliminating all potential risks of bias is impossible. The limitations of 
this evaluation study are typical of other similar studies; they are discussed below. 


Surveys that rely upon self-reported information always have a risk of response bias; 
respondents may exaggerate or underestimate, may have accurate recall difficulties, and 
may report information they perceive as socially acceptable. This study relies upon two types 
of self-reported surveys: a retrospective pre/post survey and two traditional pre/post sur- 
veys. There are benefits and drawbacks to each type of survey. 


Retrospective pre/post surveys are convenient because they occur once (improving 
response rates) and the pre/post data are matched (the same participants responding to the 
preprogram survey also responded to the postprogram survey). Certain research postulates 
that a form of bias called response-shift bias is alleviated through using retrospective 
pre/post surveys. Moore and Tananis (2009) define response-shift bias as occurring “when a 
participant uses a different internal understanding of the construct being measured to com- 
plete the pretest and posttest.” Simply put, it is common for participants to overestimate 
their knowledge, skills, and behaviors before exposure to a program or intervention. Howev- 
er, after the completion of a program, (and caused by what they learned in the program) they 
may realize their knowledge, skills, and behaviors were not truly as high as they first esti- 
mated. Therefore, it is possible for participants to rate themselves lower in a posttest due to 
a shift in their frame of reference. Obviously, this can mask the actual impact of a program. 
Conversely, other research argues traditional pretest/posttest types of surveys result in less 
biased program effectiveness estimates. In a 2011 study, Nimon, Zigarmi, and Allen discov- 
ered inflated effect sizes among retrospective pre/post survey items. According to their find- 
ings, surveys with before and after items presented side by side may introduce types of bias 
including theories of change, self-presentation, and/or effort justification. Further, to re- 
solve the issues of both response-shift bias and exaggerated effect sizes Nimon et al. recom- 
mend administering traditional pre/post surveys with a retrospective pre/post survey. While 
this solution would alleviate some bias, it was not possible to initiate in this study. 


Among all surveys in this study, small sample sizes (n) are limitations. In a study, re- 
searchers use samples to draw conclusions about an entire population; therefore, adequate 
sample sizes that most accurately reflect the characteristics of a population are needed to 
validate a study. This study relies upon nonrandom samples of program participants and 
school personnel associated with the SETIS program. The 2011-2012 SETIS program cohort 
consisted of 16 participants, 14 of whom were participants in this evaluation study. Even 
though 14 is a small number of respondents, in this case it is also the entire population we 
are studying. Sample size among the Teacher and Administrator Pre and Post Surveys are a 
greater concern. Of all the surveys, the Teacher Surveys had the largest sample with a total of 
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48 Pre Survey and 28 Post Survey respondents (after data cleaning and matching by county, 
40 Pre Surveys and 27 Post Surveys responses were analyzed). The larger Pre Survey sample 
size may skew findings slightly. The Administrator Surveys provided the smallest samples. 
Prior to matching by county the Pre Survey had nine respondents and the Post had 13; after 
matching both had eight respondents. The information gathered from the school adminis- 
trators, while useful and enriching to our study, should be viewed with caution. Additionally, 
small sample sizes may have contributed to the lack of significant findings in the Teacher 
and Administrator Surveys. 


A further limitation is the inability to calculate response rates for the Teacher and 
Administrator Surveys. The program is designed to allow SETISs the opportunity to invite 
multiple teacher colleagues and school administrators to participate in the surveys. Despite 
the fact that we are unable to ascertain how many teachers and administrators were invited 
to the surveys, we feel allowing SETISs the chance to include multiple colleagues ultimately 
creates a richer dataset. Teacher colleague and school administrator pre and post surveys 
from the 2011-2012 cohorts were matched by county, not by respondent, as there was no 
mechanism in place allowing us to determine if the same people participating in the pre sur- 
veys also responded to the post surveys. However, beginning in the 2012-2013 program year, 
we will collect information that will allow us to match pre and post surveys by person. 
Matching survey results will enable next year’s study not only to measure changes in each 
group’s knowledge, skills, and behaviors, but also changes at the individual level. 
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As a whole, survey results indicate that the 2011-2012 SETIS program was successful. 


Findings suggest the greatest impact was felt among the special education technology inte- 
gration specialists (SETISs) themselves, at the level of capacity building. Results from sur- 
veys of teacher colleagues and school administrators indicate SETISs are meeting the 
expectations of both groups. Moreover, survey data support the argument that teachers and 
school administrators are leveraging SETIS more or less as they anticipated. Even with these 
promising outcomes, there are several recommendations that may improve the implementa- 
tion, utilization, and impact of this program: 


Attempt to recruit more SETIS program candidates. This cohort consisted of 16 par- 
ticipants, 14 of whom participated in the study. With ever increasing technological 
demands in classrooms, and the capacity to train 25 SETISs per year, there is poten- 
tial to expand the program. 


Encourage SETIS program candidates to conduct staff development on technology 
integration at their schools. Only two SETISs (of 14) indicated they had led formal 
staff development throughout the program year. An increase in professional devel- 
opment opportunities could encourage more teachers and administrators to incorpo- 
rate technology into their work. 


If logistically possible, consider holding more face-to-face meetings for the SETIS 
candidate group only. Several SETIS candidates suggested increasing in-person 
meetings as well as additional time with fellow SETISs. Considering the specialized 
nature of special education content, providing SETISs greater opportunities to work 
together may help them more effectively implement technology integration in class- 
rooms to benefit students with disabilities in particular. 


Set a goal of 100% awareness of the presence (or anticipated presence) of SETISs 
among school administrators in the schools where they work. Improved communica- 
tion at all program levels may encourage optimized utilization of SETIS support and 
resources. 


Promote scheduling that allows teachers and SETISs time to cocreate technology- 
integrated lesson plans. Given time to thoughtfully plan and collaborate, teachers 
may be more inclined to incorporate technological tools and resources the SETISs of- 
fer. 


For future evaluations of the SETIS program, we offer the following recommendation: 


Consider the practicality of incorporating a mechanism to track responses and re- 
sponse rates among teacher colleagues and school administrators. Currently, SETIS 
candidates invite an unknown number of teachers and administrators to complete 
surveys. Further, it is not possible to match responses from pre surveys with post 
surveys. Adding mechanisms to address both issues would improve the reliability of 
the findings in future evaluations. 
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Appendix A. Special Education Technology Integration 
Specialist (SETIS) Survey 


1. Special Education Technology Integration Specialist Survey 


Thank you for agreeing to complete the Special Education Technology Integration Specialist (SE TIS) Survey. WWVDE is 
interested in learning about your experience as a SE TIS. The survey should take no longer than 15 minutes to complete. 
Your responses are confidential and no name will ever be associated with your responses. 


The information from this survey will be used as part of an evaluation report about the impact of the SE TIS program, so 
please be thoughtful and candid in providing your responses to each item. 


The survey contains 3 major sections that are distributed across 8 pages. Do not forget to click the "done" button on the 
final page of the survey to ensure we receive your feedback. 


If you have questions about this data collection activity, please feel free to contact Andu Meharie at 
ameharie@access.k12.w.us. 


Thank you for your participation. 


2. Section 1: About You 


1. Please indicate the county in which you serve as a technology integration specialist. 
2. What is your role within your school? 


O Classroom Teacher 


C) Administrator 
O Support Staff 


) Other (please specify) 


P| 


3. If applicable, what is the programmatic level of your school? 


[| Pre-Kindergarten 
[| Elementary 

[| Middle 

[| High 

[] Other (please specify) 


| 


4. To your knowledge, does your school currently employ any other individual who has 
completed or is currently completing the TIS program? 


Ove 
O ww 


Page 1 
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By -Yea (od || 


For each of the items in Section II of the survey, you will be asked to provide TWO self-assessments of your knowledge, 
attitudes, and professional practices as they relate to technology integration. 


First, you will be asked to provide a retrospective rating of yourself prior to having participated in the SE TIS program. 


Second, you will be asked to provide a rating of yourself today, after having participated in the SE TIS program. 
4. Section II-A 


This page includes items about your knowledge, skills and efficacy with respect to technology integration. As you answe' 
each item, please take a minute to reflect upon what you learned as a result of participating in the TIS program. Then 
choose the response that best describes your knowledge/skills/efficacy BEFORE participating and AFTER participating 


in the TIS program. 


1.1 have a strong understanding of how to use/apply the resources from Thinkfinity. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before Participating in the O O O O O 


TIS program... 


After Participating in the O O O O O 


TIS program... 


2.1 have a strong understanding of how to use/apply the resources from Curriculum 
Pathways (SAS). 


Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O O 


TIS program... 


3.1 have a strong understanding of how to use/apply the resources from West Virginia 
Writes. 


Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O @ O 


TIS program... 


After participating in the O O ee O O 


TIS program... 


4. | have a strong understanding of how to use/apply the resources from Acuity. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O Cj O O 


TIS program... 


Page 2 
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5.1 have a strong understanding of how to use/apply the resources from TechSteps. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O & O 


TIS program... 


After participating in the O O O O O 


TIS program... 


6.1 am able to assess the quality and legitimacy of web resources. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O O 


TIS program... 


7. | understand the most important issues surrounding legal use/copyright regulations and 
how they relate to integrating web resources and technology into lesson plans and 


instruction. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O Cy O =) O 


TIS program... 


8.1 am able to identify the components of a URL and to ensure it is legitimate (e.g., 


protocol, host, domain, directory, port address, etc.). 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O @ O O 


TIS program... 


After participating in the O ©) O O O 


TIS program... 


9. | have a strong understanding of the core National Educational Technology Standards 


and Performance Indicators for Teachers (NETS-T). 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O O 


TIS program... 


10.1 have a strong understanding of the core National Educational Technology Standards 
and Performance Indicators for Students (NETS-S). 


Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O ¢ O @ O 


TIS program... 


After participating in the O O O O O 


TIS program... 
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11.1 understand how to integrate Web 2.0 tools into instruction (e.g., podcasting, wikis and 


blogs, social networking, etc.). 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O e O 


TIS program... 


After participating in the O O O O O 


TIS program... 


12. | integrate digital resources/tools into my work with teachers, students and 


administrators. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O & O 


TIS program... 


After participating in the O O O O C) 


TIS program... 


13. | understand how to effectively integrate technology into instruction to improve the 


quality of students’ educational experiences. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O O 


TIS program... 


14. | have a strong understanding of the county/school acceptable use policy. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O e O O O 


TIS program... 


15.1 have a strong understanding of 21st century assessment. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O O 


TIS program... 


16. | have a strong understanding of how to design and implement project-based learning 


(PBL) in the classroom. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O O 


TIS program... 
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17.1 actively use action research to assess the impact of my teaching on student learning. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O C) 


TIS program... 


5. Section II-B 


This page includes items about the extent to which you interact with others in your school/county to effectively integrate 
technology. As you answer each item, please take a minute to reflect upon how your practices have changed as a result 
of participating in the SE TIS program. Then choose the response that best describes your practices BEFORE 
participating and AFTER participating in the SE TIS program. 


1.1 model lessons that integrate technology for other teachers in the school. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O e O 


TIS program... 


After participating in the O O O O O 


TIS program... 


2. | collaborate with others within the school to effectively integrate technology into 


instruction. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O C) 


TIS program... 


3. | serve as a resource to other teachers regarding the effective use of technology in the 


school. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O C) O 


TIS program... 


4. | can effectively work with others to assess their learning and information needs (e.g., 


other teachers, students, administrators, etc.). 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the & O O O @ 


TIS program... 


After participating in the O O O O O 


TIS program... 
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5. | have an ongoing dialogue with other staff members at the school about technology 


issues and how they can be addressed. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O O 


TIS program... 
6. Section II-C 


This page includes items about the extent to which policies, procedures and practices within your school/county have 
changed as a result of having a technology integration specialist. As you answer each item, please take a minute to 
reflect upon how your county/school has or has not changed as a result of your participation in the SE TIS program. Then 
choose the response that best describes your county/school BEFORE participating and AFTER participating in the SE 


TIS program. 


1. The school has set a time and place where staff meet in professional communities of 


practice to discuss how to effectively integrate technology into instruction. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O ©) O O O 


TIS program... 


2. Teachers at the school have time to co-plan and/or co-teach lessons that integrate 


technology into instruction. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O '® O O 


TIS program... 


3. Staff at the school understand the acceptable use policy. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O '@ O 


TIS program... 


4. Staff at the school understand key concepts and best practices regarding web literacy. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O O 


TIS program... 
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5. Staff at the school understand the most important issues surrounding legal 
use/copyright regulations and how they relate to integrating web resources and 


technology into lesson plans and instruction. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O €) O O 


TIS program... 


6. Staff at the school are familiar with the technology components of the school's strategic 


plan. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the OQ O O O O 


TIS program... 


After participating in the O O O Cy O 


TIS program... 


7. The school has policies and procedures in place that support the use of Project Based 
Learning (PBL). 


Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O @ O 


TIS program... 


After participating in the O O O O O 


TIS program... 


8. The school has policies and procedures in place that support the use of technology 


resources (e.g., Thinkfinity, Acuity, West Virginia Writes, etc.). 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O @ O 


TIS program... 


After participating in the O O O O O 


TIS program... 


9. Staff at the school regularly use Thinkfinity and or Curriculum Pathways (SAS) as a 


resource in the classroom. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O @ O O 


TIS program... 


After participating in the (e O O O O 


TIS program... 
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10. Staff at the school regularly use West Virginia Writes or another online writing program 


as a formative assessment of student writing. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O 'e O O 


TIS program... 


11. The school has a plan in place to support the implementation of TechSteps. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O O 


TIS program... 


12. Staff at the school regularly use TechSteps as part of their core content instruction. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O O 


TIS program... 


13. Staff at the school understand the core National Education Technology Standards and 


Performance Indicators for Teachers (NETS-T). 


Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O O O 


TIS program... 


14. Staff at the school understand the National Education Technology Standards and 
Performance Indicators for Students (NETS-S). 


Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 


After participating in the O O O @ O 


TIS program... 


15. Staff at the school understand West Virginia's 21st Century Skills and Tools. 


Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the & O O & O 


TIS program... 


After participating in the O O O O C) 


TIS program... 
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16. Staff at the school use Web 2.0 tools for collaboration and instruction. 
Strongly Agree 


Strongly Disagree Disagree Neutral Agree 
Before participating in the O O O O O 
TIS program... 
After participating in the O O O O O 
TIS program... 


17. Staff at the school frequently integrate digital resources/tools in their teaching. 
Neutral Agree Strongly Agree 


Strongly Disagree Disagree 
Before participating in the O O O O O 
TIS program... 
After participating in the O O O ‘. O 
TIS program... 


7. Section II-D 


This page includes items about the extent to which you have the resources necessary to accomplish technology 
integration at your school. As you answer each item, please take a minute to reflect upon the resources that were 
available to you BEFORE and AFTER participating in SE TIS program. As with the previous sections, choose the 
response that best describes your resources BEFORE participating and AFTER participating in the TIS program. 


1. | have access to a variety of high quality technology resources at my school (e.g., 


computer, digital camera, whiteboard, etc.). 
Agree Strongly Agree 


Neutral 


sto aeter ron 6 5 ro 
O O ‘ O O 


After participating in the 


Before participating in the 


TIS program... 
2. | participated in a variety of professional development about integrating technology 


resources into instruction. 
Neutral Agree Strongly Agree 


Strongly Disagree Disagree 


Before participating in the O 


TIS program... 
After participating in the O O O O O 
TIS program... 


3.1 feel | am able to direct staff in my school toward high quality technology resources that 


are relevant to their information needs. 
Disagree Neutral 


Strongly Disagree 
Before participating in the O O O O O 
TIS program... 


After participating in the 
TIS program... 


Agree Strongly Agree 
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4. | have the ability to develop useful technology resources for my school that address our 


information needs. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 
TIS program... 

After participating in the O O O O © 
TIS program... 

5. | know where to find useful and high quality technology resources that can be 


integrated into instruction. 
Strongly Disagree Disagree Neutral Agree Strongly Agree 


Before participating in the O O O O O 


TIS program... 
After participating in the O O O O O 


TIS program... 


8. Section Ill 


VWODE is interested in your feedback about your experience as a Technology Integration Specialist. Please respond to 
the following items. 


1. During the past year, did you conduct any formal staff development related to 
technology integration at your school? 


2. If you answered yes above, please describe the development opportunities you 
provided. What was the topic? Who was the audience? 


4 


3. What would you consider to be the most valuable aspects of participating in the TIS 


program? 


z 


4. What would you change about the TIS program if you could? 


4 
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5. What structures are in place at your school that allow teachers to work meaningfully 
with the TIS (e.g., common planning time, support from administration, etc.)? 


; 


6. In what ways did you work with the administration at your school to address 
technology-related issues or concerns during the past year? 


7. What barriers, if any, did you encounter in implementing what you learned as part of the 


TIS program? 


9. Thank You for Completing our Survey. 


Please click "done" to submit your survey to us. 
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Teacher Pre Survey 


Special Education Technology Integration Specialist (SE TIS) 
Program Teacher Survey (2011) 


This survey is intended to serve as an assessment of your prior integration of technology into your 
instruction and the ways in which you intend to utilize the Special Education Technology Integration 
Specialist (SE TIS) at your school. It is important that you spend time to reflect on each question and 
honestly consider the extent to which each statement is true of your past practice. Please note that 
you will receive a second survey near the end of the school year to help WVDE better understand 
how you have leveraged the services of the SE TIS in your school and what impact the SE TIS has had 
on your instructional practices. This information will only be used to evaluate the SE TIS program. All 
information is anonymous and will only be reported at the aggregate level. 


In which County do you currently work? 
Barbour Ba 


Please indicate the name of your school. 


What is your role in your school? 
Teacher 
Administrator 
Aide 
Support Staff 
Other: 


Are the majority of your students taught in the special education or regular 
education environment? 


Special education 
General education 


| teach an equal proportion of both 


How many years of experience do you have as a teacher? 
0 to 1 year 
1to 5 years 
6 to 10 years 
11 to 15 years 


more than 15 years 
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How long have you been co-teaching with your SE TIS? 
) We've never co-taught together before. 
Less than 1 year 
1 to 2 years 
2 to 4 years 
More than 4 years 


Not Applicable 


64 | The West Virginia Special Education Technology Integration Specialist (SETIS) Program 


Appendix B. SETIS Teacher Pre Survey and Post Survey 


Section |: Facilitating and Inspiring 
Student Learning and Creativity 


Please indicate the extent to which the following statements are true of yourself. 
Please think about the most recently completed school year as you respond to each 
item. 


Almost 


Not usually Sometimes Often 
always | 


| use information 
from digital sources 
to promote learning 
and engage students 
in classroom 
activities. 

| require my students 
to use digital 
resources and tools 
for writing, 
collaboration, 
reflection, research, 
and other 
assignments. 

| engage my students 
in real-world issues 
and authentic 
problem-solving. 

| require my students 
to gather information 
from sources other 
than their textbooks 
in order to complete 
their daily 
assignments (e.g., 
podcasts, videos, etc.). 
| require my students 
to present 
information and 
actively teach content 
to their fellow 
students and/or 
community members. 
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Section Il: Designing and Developing 
Digital-Age Learning Experiences and 


Assessments 


Please indicate the extent to which the following statements are true of yourself. 
Please think about the most recently completed school year as you respond to each 
item. 


Almost 


Not usually Sometimes Often 
always 


| use a blend of both 

face-to-face and 
online environments #) ( 
to deliver instruction 
to my students. 
| design and/or 
implement projects 
that emphasize 
creative thinking and 
require students to 
engage in problem- 
solving, decision- 
making and 
experimental inquiry, 
using digital 
resources/tools when 
appropriate. 
| often design and/or 
utilize student- 
centered formative 
and performance- 
based assessments 
using available digital 
resources and tools 
(e.g., WV Writes, 
Acuity, TechSteps, 
etc.), 
| use digital resources 
and tools to make 
assignments for 
students that are 
based upon their 
individual interests, 
abilities and learning 
needs. 
| require my students 
to set personal 
learning goals and to 
self-assess their 
progress toward 
meeting those goals, 
using digital 
resources and tools 
when available and 
appropriate. 
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Section III: Modeling Digital-Age Work 


and Learning 


Please indicate the extent to which the following statements are true of yourself. 
Please think about the most recently completed school year as you respond to each 
item. 


Almost 


Not usually Sometimes Often 
always 


! use digital resources 
and tools to 
communicate with 
students. 
| use digital resources 
and tools to 
communicate with my 
peers. 
| use digital resources 
and tools to 
communicate with 
parents and the 
community outside of 
my school. 
| customize the 
available digital 
resources and tools 
such as WV Writes 
(formerly Writing € @ 
Roadmap), Acuity, 
TechSteps, etc. to 
personalize learning 
for my students. 
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Section IV: Promoting and Modeling 
Digital Citizenship and Responsibility 


Please indicate the extent to which the following statements are true of yourself. 
Please think about the most recently completed school year as you respond to each 
item. 


Almost 


Not usually Sometimes Often 
always 


| advocate, model and 
teach my students 
about safe, legal and 
ethical use of digital 
information and 
technology, including 
respect for copyright, 
intellectual property, 
and the appropriate 
documentation of 
sources. 

| offer students 


opportunities to use 
digital resources and 


tools to participate in 
collaborative projects 
with students of other 
cultures that address 
current problems, 

issues or themes. 
Students in my class 
model appropriate 

online behavior and 
| social interaction 
| through digital 
activities in my 
| classroom. 
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Section V: Engaging in Professiona 
Growth and Leadership | 


Please indicate the extent to which you agree with the following statements about 
yourself, 

Please think about the most recently completed school year as you respond to each 
item. 


Almost 


Not usually Sometimes — Often 
always 


| participate actively 
in local communities 
of practice with my 
fellow teachers, either 
online or face-to-face. 
| participate actively 
in online/global 
communities with 
other educators 
outside of my local 
community to gather 
and discuss resources 
and ideas related to 


student achievement 
and learning. 
| model and teach 
other educators to use 
digital tools and 
resources to promote 
student achievement 
and learning. 
| regularly seek out 
digital resources, 
tools and research 
and evaluate its r) ® 
quality and relevance 
prior to using it in the 
classroom. 
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Section VI: Open-Ended Items 


What do you think is the role of the SE TIS? 


What do you hope to accomplish by working with an SE TIS this year? 


Please provide any additional comments you may have about the SE TIS program. 
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Teacher Post Survey 


Welcome to the SE TIS Teachers’ Post Survey 


The West Virginia Department of Education (WVDE) is interested in your feedback regarding the special education 
technology integration specialist (SE TIS) program. Your input is important to the SE TIS program. 


This survey is intended to serve as an assessment of your integration of technology into your instruction and the ways in 
which you have utilized the SE TIS at your school. It is important that you spend time to reflect on each question and 
honestly consider the extent to which each statement is true of your practice this school year. 


Please note that this is the second survey you have received and the surveys will help VWDE to understand how you 
have leveraged the services of the SE TIS in your school and what impact the SE TIS has had on your instructional 
practices. This information will only be used to evaluate the SE TIS program. 


The VWWDE understands your time is valuable and we have taken every measure to ensure that the survey is as brief as 
possible. The survey should only take 15-20 minutes to complete. Please provide your responses to the survey items and 
click submit on the final page. 


Please note that all of your responses are confidential and will only be presented as part of a group (not individually). If 
you have questions about this survey, please contact Andu Meharie in the VWDE Office of Research 
(ameharie@access.k12.w.us). If you have questions about the SE TIS program, please contact Valerie Wilson at the 
WDE Office of Special Programs at 304.558.2696 or vwilson@access.k12.wv.us. 


Thank you for your participation in this important data collection activity. 


PV erelt iam (ol! | 
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* 4, In which school district or county do you work? 


‘8 Institutional Programs O West Virginia Schools for the Deaf and 


Blind 
O Jackson 

O Wyoming 
O Jefferson 


O Out of state 


* 2. Please indicate the name of your school? 


~ 


¥ 


*3, What is your primary role within your school? 


O Teacher 
O Administrator 
O Aide 

O Support Staff 
O Other: 


Other (please specify) 
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PV srolti am (ol! | 


* 4, Are the majority of your students taught in the special education or regular education 
environment? 


'@ Special education 
O General education 


Cy | teach an equal proportion of both 


*5, How many years of experience do you have as a teacher? 


O Less than 1 year 
O 1 to 5 years 

O 6 to 10 years 

O 11 to 15 years 
O 16 or more years 
'@. | have never taught 


* 6, How long have you been co-teaching with your SE TIS? 


O We've never co-taught together before. 


O Less than 1 year 
Cy 1 to 2 years 

O 2 to 4 years 

O More than 4 years 
O Not applicable 


Section |: Facilitating and Inspiring Student Leaming and Creativity 


Please indicate the extent to which the following statements are true of you. 
Please think about the most recently completed school year as you respond to each item. 
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7. Please indicate whether you would (1) "Not Usually" to (5) "Almost Always" do the 
following: 


1 Not Usually 2 Sometimes 3 Often 4 Almost always 


| use information from digital sources to promote learning and engage O 


students in classroom activities. 


| require my students to use digital resources and tools for writing, O O 


collaboration, reflection, research, and other assignments. 
| engage my students in real-world issues and authentic problem-solving. 


| require my students to gather information from sources other than their 
textbooks in order to complete their daily assignments (e.g., podcasts, 
videos, etc.). 


| require my students to present information and actively teach content to 
their fellow students and/or community members. 


Section II: Designing and Developing Digital-Age Learning Experiences and ... 


Please indicate the extent to which the following statements are true of you. 
Please think about the most recently completed school year as your respond to each item. 


8. Please indicate whether you would (1) "Not Usually" to (5) "Almost Always" do the 
following: 


1 Not Usually 2 Sometimes 4 Almost always 


| use blend of both face-to-face and online environments to deliver 
instruction to my students. 


| design and/or implement projects that emphasize creative thinking and 
require students to engage in problem-solving, decision-making and 
experimental inquiry, using digital resources/tools when appropriate. 


| often design and/or utilize student-centered formative and performance- 
based assessments using available digital resources and tools (e.g., WV 
Writes, Acuity, TechSteps, etc.). 


| use digital resources and tools to make assignments for students that are 
based upon their individual interests, abilities and learning needs. 


l require my students to set personal learning goals and to self-assess their 
progress toward meeting those goals, using digital resources and tools 
when available and appropriate. 


Section III: Modeling Digital-Age Work and Learning 


Please indicate the extent to which the following statements are true of you. 
Please think about the most recently completed school year as you respond to each item. 
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9. Please indicate whether you would (1) "Not Usually" to (5) "Almost Always" do the 
following: 


1 Not Usually 2 Sometimes 3 Often 4 Almost always 


| use digital resources and tools to communicate with students. O O O 
| use digital resources and tools to communicate with my peers. O Cy O O 
| use digital resources and tools to communicate with parents and the O @ O O 


community outside of my school. 


| customize the available digital resources and tools such as VW Writes, O O O 
Acuity, TechSteps, etc. to personalize learning for my students. 


Section IV: Promoting and Modeling Digital Citizenship and Responsibility 


Please indicate the extent to which the following statements are true of you. 
Please think about the most recently completed school year as you respond to each item. 


10. Please indicate whether you would (1) "Not Usually” to (5) "Almost Always” do the 


following: 
1 Not Usually 2 Sometimes 3 Often 4 Almost always 


| advocate, model and teach my students about safe, legal and ethical O O O O 


use of digital information and technology, including respect for copyright, 
intellectual property, and the appropriate documentation of sources. 


| offer students opportunities to use digital resources and tools to O O O O 


participate in collaborative projects with students of other cultures that 
address current problems, issues or themes. 


Students in my class model appropriate online behavior and social O 6 e O 


interaction through digital activities in my classroom. 


Section V: Engaging in Professional Growth and Leadership 


Please indicate the extent to which the following statements are true of you. 
Please think about the most recently completed school year as you respond to each item. 


11. Please indicate whether you would (1) "Not Usually" to (5) "Almost Always” do the 


following: 
1 Not Usually 2 Sometimes 3 Often 4 Almost always 


| participate actively in local communities of practice with my fellow 
teachers, either online or face-to-face. 


outside of my local community to gather and discuss resources and ideas 
related to student achievement and learning. 


| model and teach other educators to use digital tools and resources to 
promote student achievement and learning. 


| participate actively in online/global communities with other educators O 


| regularly seek out digital resources, tools and research and evaluate 
their quality and relevance prior to using them in the classroom. 


Section VI: Open-Ended Questions 
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Please share your responses to the following questions. 


12. What do you think is the role of the SE TIS? 


4 


13. What did you accomplish by working with a SE TIS this year? 


4 


14. Please provide any additional comments you may have about the SE TIS program. 


: 
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Administrator Pre Survey 


Special Education Technology Integration Specialist (SE TIS) 
Administrative Survey (2011) 


This survey is intended to help you to determine the most effective ways to utilize the SE TIS 
in your school. Please note that you may receive a scond survey near the end of the school year to 
help WVDE better understand how school administrators are leveraging the resources provided to 
schools through the SE TIS program. 


Section I: About you 


In which county is your school located? 
Barbour [yr] 


Please indicate the name of your school. 


What is your role within your school? 
Principal 
Assistant Principal 


Other: 


Before receiving this survey, | was aware that my school would 
have a SE TIS for the 2011-12 academic year. 


Yes 


No 
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Section Il: Your plans to use the SE TIS. 
Please indicate how likely it is that you will use the SE TIS for each 
of the following purposes. 


| plan to ask my SE TIS to share what he/she has learned by 
leading standards-based professional development for the other 
teachers in my school. 


1 $2 3 4 ~°5 


Unlikely ) ) Likely 


| plan to ask my SE TIS to model the integration of technology for 
their co-teachers and others within the school. 


123 4 5 


Unlikely Likely 


| anticipate asking my SE TIS to assist me in developing the 
school's strategic plan with regard to information and technology 
needs. 


Unlikely } ) Likely 


| expect that my SE TIS will assist his/her co-teachers in 
customizing available digital resources and tools such as West 
Virginia Writes (formerly Writing Roadmap), TechSteps, and 
Acuity to personalize learning for students. 


1 2 3B A 5 


Unlikely Likely 


| will request that the SE TIS work with teachers to identify digital 
resources and tools that effectively integrate technology into 
their current curriculum. 


12 3 4 § 
Unlikely F Likely 


| will ask the TIS to conduct analyses of student data and engage 
in action research to help me understand the impact of 
technology integration in my school. 


1 2 3 4 5 


Unlikely © Likely 
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Please articulate what you believe the role of the SE TIS should be 
in your school, 


What outcomes do you expect as a result of having a SE TIS in 
your school? 


Please provide any additional comments you may have about the 
SE TIS program. 


(«Back || submit | 
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Administrator Post Survey 


Welcome to the SE TIS Administrators’ Post Survey 


The VWWDE is interested in your feedback regarding the special education technology integration specialist (SE TIS) 
program. Your input is very important to the SE TIS program. 


This survey is intended to determine effective ways to utilize a SE TIS in schools. Please note that this is the second 
survey you are receiving to help VWVDE understand how school administrators are leveraging the resources provided to 
schools through the SE TIS program. 


The VWWDE understands your time is valuable and we have taken every measure to ensure that the survey is as brief as 
possible. The survey should only take 10-15 minutes to complete. Please provide your responses to the survey items and 
click submit on the final page. 


Please note that all of your responses are confidential and will only be presented as part of a group (not individually). If 
you have questions about this survey, please contact Andu Meharie in the VWW/DE Office of Research 
(ameharie@access.k12.w.us). If you have questions about the SE TIS program, please contact Valerie Wilson at the 
VWDE Office of Special Programs at 304.558.2696 or vwilson@access.k12.wv.us. 


Thank you for your participation in this important data collection activity. 


About You 
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* 4, In which school district or county do you work? 


O Institutional Programs O West Virginia Schools for the Deaf and 


Blind 
O Jackson 

O Wyoming 
O Jefferson 


O Out of state 


* 2. Please indicate the name of your school? 


*3, What is your primary role within your school? 


Cy Principal 
O Assistant Principal 
O Other 


Other (please specify) 


Your Plans to Use the SE TIS 
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Please indicate, on a scale of 1 to 5, how accurately each statement reflects the activities your SE TIS during this school 


year. 
4. Based on your classroom observations this school year, has the SE TIS candidate 
increased student engagement in curricular activities through the increased use of 


technology? 


5. My SE TIS shared what he/she learned by leading professional development for the 


other teachers in my school. 


6. My SE TIS modeled the integration of technology for her/his co-teachers and others 


within the school. 


7. My SE TIS assisted me in developing the school’s strategic plan with regard to 


information and technology needs. 
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8. My SE TIS assisted his/her co-teachers in customizing available digital resources and 
tools such as West Virginia Writes, TechSteps, Acuity, Edmodo, Thinkfinity, etc. to 
personalize learning for students. 


9.1 requested that the SE TIS work with teachers to identify digital resources and tools that 
effectively integrate technology into their current curriculum. 


10. | asked the SE TIS to conduct analyses of student data and engage in action research 
to help me understand the impact of technology integration in my school. 


11. Please articulate what you believe the role of the SE TIS should be in your school. 


| 


12. What outcomes did you observe as a result of having a SE TIS in your school? 


‘ 


13. Please provide any additional comments you may have about the SE TIS program. 


4 
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Table 22. Human Capacity Items: Pre and Post Means, T Test and Effect Size 


Significance Cohen’s Effect 


Pre Post of difference d size 
Item Item Statement n Mean SD Mean SD t p 
1 | havea strong understanding of 14 2.07 1.14 4.43 0.65 8.75 <.0001 2.64 very 
how to use/apply the resources large 


from Thinkfinity. 
2 | havea strong understanding of 14 1.93 1.21 3.71 1.07 5.96 <.0001 1.63 very 


how to use/apply the resources large 
from Curriculum Pathways 
(SAS). 

3 | have a strong understanding of 13 2.77 1.36 4.31 0.75 4.63 .0006 1.45 very 
how to use/apply the resources large 


from West Virginia Writes. 
4 |havea strong understanding of 13 2.54 1.20 4.00 0.78 5.73 <.0001 1.51 very 


how to use/apply the resources large 
from Acuity. 

5 | havea strong understanding of 14 2.21 1.05 4.21 0.80 6.75 <.0001 2.22 very 
how to use/apply the resources large 


from TechSteps. 
6 | am able to assess the quality 14 3.21 1.05 4.57 0.51 6.82 <.0001 1.70 very 


and legitimacy of web large 
resources. 

7 | understand the most 14 2.07 1.14 4.21 0.58 9.28 <.0001 2.46 very 
important issues surrounding large 


legal use/copyright regulations 
and how they relate to 
integrating web resources and 
technology into lesson plans 
and instruction. 

8 | am able to identify the 14 2.64 1.55 4.14 0.77 4.84 .0003 1.27 very 
components of a URL and to large 
ensure it is legitimate (e.g., 
protocol, host, domain, 
directory, port address, etc.). 

9 | havea strong understanding of 14 1.79 0.70 4.07 0.62 10.36 <.0001 3.60 very 
the core National Educational large 
Technology Standards and 
Performance Indicators for 
Teachers (NETS-T). 

10 | havea strong understanding of 14 1.79 0.70 4.00 0.55 10.33 <.0001 3.64 very 
the core National Educational large 
Technology Standards and 
Performance Indicators for 
Students (NETS-S). 


Table 22 continues next page 
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Table 22. |. Human Capacity Items: Pre and Post Means, T Test and Effect Size 
Significance Cohen’s Effect 
Pre Post of difference d size 
Item Item Statement n Mean SD Mean SD t p 

11 | understand how to integrate 14 1.86 0.86 4.43 0.65 9.47 <.0001 3.50 very 
Web 2.0 tools into instruction large 
(e.g., podcasting, wikis and 
blogs, social networking, etc.). 

12 | integrate digital 14 2.36 1.22 4.64 0.50 8.00 <.0001 2.55 very 
resources/tools into my work large 
with teachers, students and 
administrators. 

13 | understand how to effectively 14 2.50 1.16 4.79 0.43 7.51 <.0001 2.71 very 
integrate technology into large 
instruction to improve the 
quality of students' educational 
experiences. 

14 | havea strong understanding of 14 2.86 1.03 4.43 0.65 4.58  .0005 1.90 very 
the county/school acceptable large 
use policy. 

15 | havea strong understanding of 14 2.29 1.07 4.21 0.70 5.69 <.0001 2.22 very 
21st century assessment. large 

16 | havea strong understanding of 13 2.50 1.02 4.00 1.00 5.20 .0002 1.54 very 
how to design and implement large 
project-based learning (PBL) in 
the classroom. 

17 lactively use actionresearchto 14 2.14 0.86 3.79 0.97 5.34 .0001 1.85 very 
assess the impact of my large 


teaching on student learning. 
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Table 23. Organizational Capacity Items: Pre and Post Means, T Test and Effect Size 


Significance of 


Pre Post difference Cohen’s Effect 
Item Item Statement n Mean SD Mean SD t p d size 

18 | model lessons that 14 2.36 1.15 4.07 4.07 4.84 0.000 1.93 very 
integrate technology for 3 large 
other teachers in the 
school. 

19 | collaborate with others 14 2.29 1.14 4.36 4.36 6.79 <.000 2.45 very 
within the school to 1 large 
effectively integrate 
technology into 
instruction. 

20 | serve as a resource to 13. 2.86 1.17 454 4.54 6.88 <.000 1.91 very 
other teachers regarding 1 large 
the effective use of 
technology in the school. 

21 | can effectively work with 13 2.64 1.15 4.15 4.15 4.62 0.000 1.72 very 
others to assess their 6 large 
learning and information 
needs (e.g., other teachers, 
students, administrators, 
etc.). 

22 | have an ongoing dialogue 14 2.57 0.94 4.36 4.36 6.36 <.000 2.19 very 
with other staff members 1 large 


at the school about 
technology issues and how 
they can be addressed. 
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Table 24. Structural Capacity Items: Pre and Post Means, T Test and Effect Size 


Pre 


Post 


Significance 
of difference 


Item Item Statement n Mean 


SD 


Mean 


SD 


t 


p 


Cohen’s Effect 
d size 


23 The school has set a time 14 
and place where staff 
meet in professional 
communities of practice to 
discuss how to effectively 
integrate technology into 
instruction. 

24 Teachers at the school 14 
have time to co-plan 
and/or co-teach lessons 
that integrate technology 
into instruction. 

25 Staff at the school 14 
understand the acceptable 
use policy. 

26 Staff at the school 14 
understand key concepts 
and best practices 
regarding web literacy. 

27 Staff at the school 13 
understand the most 
important issues 
surrounding legal 
use/copyright regulations 
and how they relate to 
integrating web resources 
and technology into lesson 
plans and instruction. 

28 Staff at the school are 14 
familiar with the 
technology components of 
the school's strategic plan. 

29 The school has policies 14 
and procedures in place 
that support the use of 
Project Based Learning 
(PBL). 

30 The school has policies 14 
and procedures in place 
that support the use of 
technology resources (e.g., 
Thinkfinity, Acuity, West 
Virginia Writes, etc.). 


2.14 


2.43 


3.21 


2.71 


2.85 


2.93 


2.64 


3.64 


1.17 


1.22 


0.97 


0.91 


0.90 


1.07 


1.08 


0.84 


3.00 


3.29 


3.79 


3.57 


3.38 


3.36 


3.50 


4.21 


1.36 


1.44 


0.70 


0.65 


0.96 


0.84 


0.85 


0.80 


3.12 


2.92 


2.28 


3.38 


2.50 


2.48 


3.71 


2.28 


.0081 


.0120 


.0401 


.0049 


.0279 


.0275 


.0026 


.0401 


0.70 medium 


0.67 medium 


0.70 medium 


1.12 very 
large 


0.60 medium 


0.46 medium 


0.91 large 


0.72 medium 
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Table 24. 


Appendix D. SETIS Survey Capacity Indices 


Structural Capacity Items: Pre and Post Means, T Test and Effect Size 


Item 


Item Statement 


Pre 


Post 


Significance 
of difference 


n Mean 


SD 


Mean 


SD 


t 


p 


Cohen’s Effect 
d size 


31 


32 


33 


34 


35 


36 


37 


38 


39 


Staff at the school 
regularly use Thinkfinity 
and or Curriculum 
Pathways (SAS) as a 
resource in the classroom. 
Staff at the school 
regularly use West Virginia 
Writes or another online 
writing program as a 
formative assessment of 
student writing. 

The school has a plan in 
place to support the 
implementation of 
TechSteps. 

Staff at the school 
regularly use TechSteps as 
part of their core content 
instruction. 

Staff at the school 
understand the core 
National Education 
Technology Standards and 
Performance Indicators for 
Teachers (NETS-T). 

Staff at the school 
understand the National 
Education Technology 
Standards and 
Performance Indicators for 
Students (NETS-S). 

Staff at the school 
understand West Virginia's 
21st Century Skills and 
Tools. 

Staff at the school use 
Web 2.0 tools for 
collaboration and 
instruction. 

Staff at the school 
frequently integrate digital 
resources/tools in their 
teaching. 


14 


14 


14 


14 


14 


14 


14 


14 


14 


2.00 


3.43 


3.21 


2.50 


2.14 


2.21 


2.43 


2.36 


2.50 


0.68 


0.85 


1.12 


1.22 


0.86 


0.80 


1.02 


1.08 


0.94 


2.79 


4.21 


3.86 


3.29 


3.00 


2.93 


3.50 


3.36 


3.79 


0.97 


0.80 


1.10 


0.99 


0.96 


1.00 


0.94 


0.93 


0.89 


4.20 .0010 


4.20 


2.59 


2.62 


3.38 


2.92 


4.02 


3.89 


4.22 


.0010 


.0224 


.0212 


.0049 


.0118 


.0015 


.0018 


.0010 


0.97 large 


0.99 


0.60 


0.73 


0.97 


0.82 


1.14 


1.03 


1.46 


large 


medium 


medium 


large 


large 


very 
large 


large 


very 
large 
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Appendix D. SETIS Survey Capacity Indices 


Table 25. Material Capacity Items: Pre and Post Means, T Test and Effect Size 
Significance of 
Pre Post difference Cohen’s Effect 
Item Item Statement n Mean SD Mean SD t p d size 

40 | have access to a variety 13. 3.85 0.80 4.54 0.52 2.63 0.0218 1.07 large 
of high quality technology 
resources at my school 
(e.g., computer, digital 
camera, whiteboard, etc.). 

41 | participated in a variety 13. 2.771.011 4.23 0.73 6.01 <.0001 1.73 very 
of professional large 
development about 
integrating technology 
resources into instruction. 

42 |feell am able to direct 13 2.08 0.86 4.23 0.73 7.87 <.0001 2.81 very 
staff in my school toward large 
high quality technology 
resources that are relevant 
to their information needs. 

43 Ihave the ability to 13 1.92 0.86 4.15 0.69 8.68 <.0001 2.98 very 
develop useful technology large 
resources for my school 
that address our 
information needs. 

44 | know where to find 13 2.23 = =—-0.83 454 0.52 8.78 <.0001 3.46 very 
useful and high quality large 


technology resources that 
can be integrated into 
instruction. 
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Appendix E. Teacher Colleague Technology Use Indices 


Table 


26. Facilitate Items: Pre and Post Means, T Test and Effect Size 


Item 


Item Statement 


Pre 


Post 


Significance of 
difference 


n 


Mean 


SD 


n 


Mean 


| use information from digital 
sources to promote learning 
and engage students in 
classroom activities. 

| require my students to use 
digital resources and tools 
for writing, collaboration, 
reflection, research, and 
other assignments. 

| engage my students in real- 
world issues and authentic 
problem-solving. 

| require my students to 
gather information from 
sources other than their 
textbooks in order to 
complete their daily 
assignments (e.g., podcasts, 
videos, etc.). 

| require my students to 
present information and 
actively teach content to 
their fellow students and/or 
community members. 


40 


40 


40 


40 


40 


2.70 


2.03 


2.55 


2.08 


2.00 


0.91 


1.00 


0.81 


0.89 


0.93 


26 


26 


26 


26 


26 


2.92 


2.46 


2.92 


2.46 


2.23 


0.74 


0.95 


0.84 


0.95 


0.91 


1.77 


1.79 


1.68 


1.00 


0.08 


0.08 


0.10 


0.33 


NOTE: 


Cohen’s d effect sizes were not computed for any nonsignificant individual item results. 
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Table 27. Design Items: Pre and Post Means, T Test and Effect Size 


Significance 
Pre Post of difference 
Item Item Statement n Mean SD n Mean SD t p 
1 | use a blend of both face-to-face and 40 2.13 0.91 25 2.08 0.95 0.19 0.85 


online environments to deliver 
instruction to my students. 
2 | design and/or implement projects that 40 2.18 0.90 25 2.52 0.87 1.52 0.13 
emphasize creative thinking and require 
students to engage in problem-solving, 
decision-making and experimental 
inquiry, using digital resources/tools 
when appropriate. 
3 | often design and/or utilize student- 40 2.18 1.03 25 2.32 0.85 0.59 0.56 
centered formative and performance- 
based assessments using available 
digital resources and tools (e.g., WV 
Writes, Acuity, TechSteps, etc.). 
4 | use digital resources and tools to make 40 2.28 0.99 25 2.60 0.87 1.35 0.18 
assignments for students that are based 
upon their individual interests, abilities 
and learning needs. 
5 | require my students to set personal 40 1.95 0.96 25 2.08 0.76 0.57 0.57 
learning goals and to self-assess their 
progress toward meeting those goals, 
using digital resources and tools when 
available and appropriate. 


NOTE: Cohen’s d effect sizes were not computed for any nonsignificant individual item results. 


Table 28. Model Items: Pre and Post Means, T Test and Effect Size 


Significance 

Pre Post of difference 

Item Item Statement n Mean SD n Mean SD t p 

1 | use digital resources and tools to 40 2.03 1.07 25 2.08 1.04 0.20 0.84 
communicate with students. 

2 | use digital resources and tools to 40 2.85 0.80 25 3.16 0.80 1.52 0.13 
communicate with my peers. 

3 | use digital resources and tools to 40 2.55 0.99 25 2.44 # 0.96 0.44 0.66 


communicate with parents and the 
community outside of my school. 
4 | customize the available digital 40 2.10 0.98 25 2.29 1.20 0.70 0.49 
resources and tools such as WV Writes, 
Acuity, TechSteps, etc. to personalize 
learning for my students. 


NOTE: Cohen’s d effect sizes were not computed for any nonsignificant individual item results. 
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Appendix E. Teacher Colleague Technology Use Indices 


Table 29. Promote Items: Pre and Post Means, T Test and Effect Size 


Pre 


Post 


Significance of 
difference 


Item Item Statement n Mean 


SD 


n Mean 


SD 


1 | advocate, model and teach my 40 2.48 
students about safe, legal and ethical 
use of digital information and 
technology, including respect for 
copyright, intellectual property, and 
the appropriate documentation of 
sources. 
2 | offer students opportunities to use 40 1.75 
digital resources and tools to 
participate in collaborative projects 
with students of other cultures that 
address current problems, issues or 
themes. 
3 Students in my class model appropriate 40 2.63 
online behavior and social interaction 
through digital activities in my 
classroom. 


1.06 


1.01 


1.00 


25 = 2.84 


25 1.92 


25 2.80 


1.03 


1.00 


0.91 


0.67 


0.71 


0.51 


0.48 


NOTE: Cohen’s d effect sizes were not computed for any nonsignificant individual item results. 


Table 30. Engage Items: Pre and Post Means, T Test and Effect Size 


Pre 


Post 


Significance 
of difference 


Item Item Statement n Mean 


SD 


n Mean 


SD 


t 


p 


1 | participate actively in local 40 2.48 
communities of practice with my 
fellow teachers, either online or face- 
to-face. 
2 | participate actively in online/global 40 1.75 
communities with other educators 
outside of my local community to 
gather and discuss resources and ideas 
related to student achievement and 
learning. 
3 | model and teach other educators to 40 1.83 
use digital tools and resources to 
promote student achievement and 
learning. 
4 | regularly seek out digital resources, 40 2.35 
tools and research and evaluate its 
quality and relevance prior to using it 
in the classroom. 


0.96 


0.78 


0.78 


1.03 


24 = 2.42 


24 1.79 


24 =1.63 


24 = 2.33 


0.88 


0.93 


0.65 


0.96 


0.24 


0.19 


1.06 


0.06 


0.81 


0.85 


0.30 


0.95 


NOTE: Cohen’s d effect sizes were not computed for any nonsignificant individual item results. 


The West Virginia Special Education Technology Integration Specialist (SETIS) Program | 93 


94 


Appendix F. School Administrator Survey Results 


Table 31. SETIS School Administrator Survey Items: Pre and Post Means, T Test and Effect Size 


Signifi-cance of 


Pre Post differ-ence 
Item Item Statement n Mean SD Mean SD t p 
Share | plan to ask my SETIS to share/ My SETIS 8 4.50 0.53 4.38 0.74 0.39 0.71 


shared what he/she has learned by 
leading professional development for the 
other teachers in my school. 
Model _ | plan to ask my SETIS to model/ My SETIS 8 4.38 0.74 4.63 0.52 0.78 0.44 
modeled the integration of technology 
for his/her coteachers and others within 
the school. 
Assist in | anticipate asking my SETIS to assist/ My 8 4.38 0.92 3.88 0.83 1.14 0.27 
plan SETIS assisted me in developing the 
school’s strategic plan with regard to 
information and technology needs. 
Assist | expect that my SETIS will assist/ My 8 4.50 0.76 4.38 0.74 0.33 0.74 
teacher SETIS assisted his/her coteachers in 
customizing available digital resources 
and tools such as WV Writes, TechSteps, 
Acuity, Edmodo, Thinkfinity, etc. to 
personalize learning for students. 
Work | will request that the SETIS/ | requested 8 4.63 0.74 3.88 1.46 1.30 0.22 
with that the SETIS work with teachers to 
teacher identify digital resources and tools that 
effectively integrate technology into their 
current curriculum. 
Conduct | will ask the SETIS/ | asked the SETIS to 8 4.00 1.20 3.63 1.30 0.60 .056 
analysis conduct analyses of student data and 
engage in action research to help me 
understand the impact of technology 
integration at my school. 
Observe Based on your classroom observations 8 NA NA 4.75 0.46 NA NA 
this school year, has the SETIS candidate 
increased student engagement in 
curricular activities through increased 
use of technology. 


NOTE: Cohen’s d effect sizes were not computed for any nonsignificant individual item results. 
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